A robust cross-domain adaptive classification method based on denoising contrastive learning

Abstract

The present invention discloses a robust cross-domain adaptive classification method based on denoising contrastive learning, comprising the following steps: obtaining source domain data and target domain data; selecting clean samples from the source domain and target domain data using a classification network; establishing a graph structure based on all samples in the source domain and target domain data, performing label propagation on the noisy samples according to the graph structure to purify them to obtain robust label matrices corresponding to the source domain and target domain data, and constructing a target loss function based on the robust label matrix; performing a cross-domain denoising contrastive learning method including cross-domain intra-class alignment and inter-class dispersion on the clean samples; constructing a distribution matching function based on the cross-domain intra-class alignment and an embedding discrete function based on the inter-class dispersion; constructing a final target optimization function based on the target loss function, the distribution matching function, and the embedding discrete function; and iteratively implementing classification of the target domain data based on the classification network and the target optimization function. The present invention makes supervision more reliable and enhances the ability to distinguish between classes.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a robust cross-domain adaptive classification method based on denoising contrast learning. Background Art

[0002] Unsupervised domain adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. UDA reduces the domain shift between the training (source domain) and test datasets (target domain) by minimizing the domain difference.

[0003] Existing unsupervised domain adaptation methods can be roughly divided into difference-based methods and adversarial methods. Existing difference-based methods tend to adjust the domain distribution by minimizing carefully designed statistical indicators. For example, the article "Deep domain confusion: Maximizing for domain invariance" published by Eric Tzeng et al. proposed a domain confusion loss based on the maximum mean discrepancy (MMD) to optimize classification and domain invariance; in order to extend semi-supervised learning technology to unsupervised domain adaptation problems, the article "Label propagation with augmented anchors: A simple semi-supervised learning baseline for unsupervised domain adaptation" published by Zhang Yabin et al. proposed a label propagation algorithm with augmented anchors (A 2 LP) improves the quality of target pseudo-labels by generating unlabeled virtual instances; existing adversarial methods learn domain-invariant features through a minimum-maximum game. For example, the article "Unsupervised domain adaptation by backpropagation" published by Yaroslav Ganin et al. and the article "Conditional adversarial domain adaptation" published by Long Mingsheng et al. introduce a domain identifier to distinguish between source and target samples, while the feature extractor attempts to confuse the domain identifier and learn domain-invariant features through an adversarial method.

[0004] However, in practical domain adaptation problems, completely clean source labels are almost impossible to achieve. The inevitable label noise in the source domain data can mislead the model during training, resulting in poor model performance and poor robustness. Therefore, source label noise remains a challenging task, and learning domain-invariant and discriminative representations that are robust to label noise remains an important research area. Although existing methods combat source label noise by designing specific denoising objectives and minimizing distribution differences, these methods ignore the potential class information between noise samples and target samples, resulting in classification confusion and poor model robustness. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a robust cross-domain adaptive classification method based on denoising contrastive learning. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] An embodiment of the present invention provides a robust cross-domain adaptive classification method based on denoising contrastive learning, including:

[0007] Acquire source domain data and target domain data; wherein the source domain data and the target domain data share the same label space; the source domain data and the target domain data respectively include clean samples and noise samples, and the source domain data are all labeled samples, and the target domain data are unlabeled samples;

[0008] Selecting clean samples from the source domain data and the target domain data respectively using a classification network;

[0009] Establishing a graph structure based on all samples in the source domain data and the target domain data, performing label propagation on noise samples based on the established graph structure to purify the noise samples to obtain robust label matrices corresponding to the source domain data and the target domain data, respectively, and constructing a target loss function based on the robust label matrices;

[0010] On the selected clean samples, a cross-domain denoising contrastive learning method, including cross-domain intra-class alignment and inter-class dispersion, is performed. A distribution matching function is constructed based on cross-domain intra-class alignment, and an embedding discrete function is constructed based on inter-class dispersion.

[0011] Constructing a final target optimization function according to the target loss function, the distribution matching function and the embedded discrete function;

[0012] The classification of the target domain data is achieved iteratively according to the classification network and the target optimization function.

[0013] In one embodiment of the present invention, selecting clean samples from the source domain data and the target domain data using a classification network includes:

[0014] Outputting cross entropy loss values ​​corresponding to all samples in the source domain data and the target domain data using a classification network;

[0015] Compare the cross entropy loss value with the confidence threshold. If the cross entropy loss value is less than or equal to the confidence threshold, select the sample corresponding to the cross entropy loss value as a clean sample; otherwise, select the sample corresponding to the cross entropy loss value as a noise sample.

[0016] In one embodiment of the present invention, a graph structure is established based on all samples in the source domain data and the target domain data, and label propagation is performed on noise samples based on the established graph structure to purify the noise samples to obtain robust label matrices corresponding to the source domain data and the target domain data, respectively, including:

[0017] Establishing corresponding graph structures according to the similarities between all samples in the source domain data and the target domain data;

[0018] Calculating the embedding distances between adjacent nodes in the graph structure to construct a corresponding adjacency matrix;

[0019] Performing label propagation iterative processing according to the adjacency matrix and the initial hard label matrices corresponding to the clean samples in the source domain data and the target domain data to obtain a soft label matrix corresponding to the noise sample;

[0020] The initial hard label matrix and the soft label matrix form a corresponding robust label matrix.

[0021] In one embodiment of the present invention, the soft label matrix formula corresponding to the noise sample in the target domain data is expressed as:

[0022]

[0023] Among them, Y t* represents the soft label matrix corresponding to the noise sample in the target domain data, I represents the identity matrix, γ represents the trade-off parameter, γ>0, represents the degree matrix of the adjacency matrix A, D represents a diagonal matrix, and the adjacency matrix A is the element A ij A collection of ij represents the embedding distance between the i-th sample and the j-th sample, represent the i-th sample and the j-th sample respectively, Representation sample The adjacent set of Represents samples sample The feature extraction function, Yt(0) represents the initial hard label matrix, represents the i-th sample The initial hard label matrix, One-hot(·) represents the pseudo-label one-hot vector corresponding to the classification network output. Represents the set of clean samples in the target domain data.

[0024] In one embodiment of the present invention, the objective loss function formula constructed according to the robust label matrix is ​​expressed as:

[0025] l robust =l sup +η·l d (D(G(X s )), D(G(X t ))),

[0026] Among them, l robust represents the target loss function, l ce (·,·) is the cross entropy loss, L(G(X s )) is to extract feature X from the source domain data S The predicted label of for The transposed vector form of Represents the initial hard label matrix Y corresponding to the source domain data s(0) and the soft label matrix The robust label matrix composed of L(G(X t )) is to extract feature X from the target domain data t The predicted label of for The transposed vector form of Represents the initial hard label matrix Y corresponding to the target domain data t(0) and the soft label matrix Y t* The robust label matrix composed of η represents the contribution parameter, l d (·,·) is the domain similarity loss, D(G(X S )) is to extract feature X from the source domain data S The domain where D(G(X t )) is to extract feature X from the target domain data t The domain where the .

[0027] In one embodiment of the present invention, performing a process including cross-domain intra-class alignment includes:

[0028] Clean samples in the source domain data and the target domain data are used as embedding prototypes to calculate cross-domain intra-class distribution matching, thereby achieving cross-domain intra-class alignment.

[0029] In one embodiment of the present invention, the distribution matching function formula constructed based on cross-domain intra-class alignment is expressed as:

[0030]

[0031] Among them, l dm represents the distribution matching function, C represents the number of classes of source domain data or target domain data, φ(·, ·) represents the calculation of the square of the Euclidean distance, represents the embedded prototype of the k-th clean sample in the source domain data, Describe the embedding prototype of the k-th clean sample in the target domain data.

[0032] In one embodiment of the present invention, the process of performing inter-class scattering includes:

[0033] Taking the clean samples in the source domain data as the object, the embedded prototype of the sample of the same category in the target domain data corresponding to the clean sample as the positive sample, and the clean sample of a different category from the clean sample as the negative sample, the discrete embedding calculations in the two domains are performed to achieve inter-class dispersion.

[0034] In one embodiment of the present invention, the embedding discrete function formula constructed based on inter-class dispersion is expressed as:

[0035]

[0036] Among them, l cl represents the embedded discrete function, represents the number of clean samples in the source domain data, represents the set of clean samples in the source domain data, sim(·,·) represents the similarity evaluation function for two items, represents the label of the i-th clean sample in the source domain data, Indicates the corresponding label in the target domain data The embedding prototype of samples of the same category, i.e. As a positive sample, Represent the negative samples in the source domain data Negative samples The feature extraction function.

[0037] In one embodiment of the present invention, the final objective optimization function formula constructed according to the objective loss function, the distribution matching function and the embedded discrete function is expressed as:

[0038] l=l robust+l dm -l cl ,

[0039] Among them, l represents the final target optimization function, l robust represents the target loss function, l dm represents the distribution matching function, l cl represents an embedded discrete function.

[0040] Beneficial effects of the present invention:

[0041] The robust cross-domain adaptive classification method based on denoising contrastive learning proposed in the present invention can simultaneously generate robust discrimination of noise samples and domain-invariant representations, specifically: obtaining source domain data and target domain data; wherein the source domain data and the target domain data share the same label space; the source domain data and the target domain data respectively include clean samples and noise samples, and the source domain data are all labeled samples, and the target domain data are all unlabeled samples; using a classification network to select clean samples from the source domain data and the target domain data respectively; establishing a graph structure based on all samples in the source domain data and the target domain data, and The noise samples are label propagated to purify the noise samples to obtain the robust label matrices corresponding to the source domain data and the target domain data respectively, and the target loss function is constructed according to the robust label matrices; on the selected clean samples, the cross-domain denoising contrastive learning method including cross-domain intra-class alignment and inter-class dispersion is performed respectively; among them, the distribution matching function is constructed based on the cross-domain intra-class alignment, and the embedding discrete function is constructed based on the inter-class dispersion; the final target optimization function is constructed according to the target loss function, the distribution matching function and the embedding discrete function; the classification network and the target optimization function are iteratively implemented to realize the classification of the target domain data. It can be seen that in addition to considering the class information of the source domain data, the embodiments of the present invention also consider extracting robust class information from the target domain data to provide more effective supervision; the embodiments of the present invention simultaneously select clean samples for the source domain data and the target domain, and filter out noise samples in the two domains through label propagation, instead of directly discarding the noise samples, thereby obtaining sufficient robust class information in the two domains, extracting robust class information to supervise the network to learn class-level discriminant representation, making supervision more reliable, thereby enhancing the class-level distinction ability; the embodiments of the present invention also propose a cross-domain denoising contrast learning method, which operates on the clean samples selected in the two domains, so as to better ensure cross-domain class invariance, improve the class-level distinction ability of the model, enhance the robustness of the representation learning process, and produce more robust classification results.

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 11 is a flowchart of a robust cross-domain adaptive classification method based on denoising contrastive learning provided by an embodiment of the present invention;

[0044] Figure 2 (a)~ Figure 2 (b) is a schematic diagram of the framework of the implementation process of the traditional method and the method of the present invention;

[0045] Figure 3 Schematic diagram of a robust cross-domain contrast adaptive network provided by an embodiment of the present invention;

[0046] Figure 4 Schematic diagram of the robust information extraction process provided by an embodiment of the present invention;

[0047] Figure 5 (a)~ Figure 5 (d) is a schematic diagram of the classification effects of DANN, RDA, and the method of the present invention on the original data;

[0048] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0050] To achieve more robust classification results, see Figure 1 , the embodiment of the present invention provides a robust cross-domain adaptive classification method based on denoising contrastive learning, and Figure 2 (a)~ Figure 2 (b) illustrates the implementation process of the traditional method and the method of the present invention. Figure 2 (b) The proposed method can solve Figure 2 The class confusion problem caused by poor robustness due to label noise and neglect of class information in (a) is solved by the following steps:

[0051] S10. Acquire source domain data and target domain data; wherein the source domain data and the target domain data share the same label space; the source domain data and the target domain data respectively include clean samples and noise samples, and the source domain data are all labeled samples, and the target domain data are unlabeled samples.

[0052] The sample set in the source domain data S obtained by the embodiment of the present invention is in and Represents the initial feature vector and label of the i-th sample, n s is the number of samples in the source domain data, d is the initial vector dimension, and the feature vector of the sample in the source domain data is recorded as The corresponding label vector is recorded as Similarly, the unlabeled sample set in the target domain data is recorded as in Represents the initial feature vector of the i-th sample in the target domain data, n t is the number of samples in the target domain data, and the feature vector of the sample in the target domain data is recorded as The source domain data and target domain data share the same label space, and the total number of classes is C. Assume that there is labeled source domain data D s and unlabeled target domain data D t Both can be used for model learning.

[0053] S20. Use the classification network to select clean samples from the source domain data and the target domain data respectively.

[0054] In the embodiment of the present invention, the classification network can select an existing network, such as Figure 3 The classification network shown consists of a feature extractor G(·), a label predictor L(·), and a domain classifier D(·). A ResNet-5 residual network pre-trained on ImageNet is used as the initialization backbone network. The feature extractor G(·) and label predictor L(·) are selected from the pre-trained ResNet network; the domain classifier D(·) has three fully connected layers, the first two of which have ReLU activation and Dropout, and the last fully connected layer has a sigmoid activation.

[0055] During the training process of the classification network, the network is optimized by minimizing the source classification error and the difference between the two domains. The objective optimization function formula corresponding to the network is expressed as:

[0056] l=l ce (L(G(X s )), Y s )+η·l d (D(G(X s )), D(G(X t ))),

[0057] in, is the cross entropy loss, is the domain similarity loss to obtain domain invariant features, and it is expected that the game between G(·) and D(·) will reach equilibrium, L(G(X s )) is to extract feature X from the source domain data S The predicted label, η represents the contribution parameter, specifically representing the contribution parameter of the domain classifier D(·) to the overall loss, D(G(X S )) is to extract feature X from the source domain data S The domain where D(G(X t)) is to extract feature X from the target domain data t The domain classifier D(·) is based on the feature X S , Feature X t Whether it is classified as source domain data or target domain data.

[0058] Furthermore, the inventors discovered that existing methods only obtain reliable supervision by selecting clean samples from the source domain data, and are unable to mine robust class information from the remaining noise samples and target samples. Unlike existing methods, the embodiments of the present invention select clean samples from both the source and target domain data. The clean samples selected from both domains can serve as reliable supervision for model learning. Specifically:

[0059] First, the embodiment of the present invention uses a classification network to select clean samples from the source domain data and the target domain data respectively, including: using the classification network to output the cross entropy loss value corresponding to all samples in the source domain data and the target domain data; comparing the cross entropy loss value with the confidence threshold, if the cross entropy loss value is less than or equal to the confidence threshold, selecting the sample corresponding to the cross entropy loss value as a clean sample, otherwise, the sample corresponding to the cross entropy loss value is a noise sample. The above selection of clean samples in the target domain data can be expressed as:

[0060]

[0061] in, represents the cross entropy loss value of the i-th sample in the target domain data, and β is the confidence threshold set in advance to determine the selected clean sample. The specific value can be designed according to actual requirements. and Represent the clean sample set and the noise sample set in the target domain data respectively. In this way, clean samples with high confidence in the target domain data can be selected, and the corresponding labels can be regarded as the robust class information of these clean samples. The clean sample selection process in the source domain data is the same as that in the target domain data, so it will not be repeated here. The clean sample set and the noise sample set in the source domain data are respectively represented as and In this way, clean samples selected in both domains can serve as reliable supervision for model learning.

[0062] S30. Establish a graph structure based on all samples in the source domain data and the target domain data, perform label propagation on the noise samples based on the established graph structure to purify the noise samples to obtain robust label matrices corresponding to the source domain data and the target domain data respectively, and construct a target loss function based on the robust label matrix.

[0063] In the source domain data and target domain data, in addition to the clean samples, the remaining noise samples still contain potential class information, which is often not detected by existing methods. Therefore, the embodiment of the present invention proposes to mine the robust class information of these noise samples for more reliable supervision. In order to extract the robust class information in the target domain data, the embodiment of the present invention converts the pseudo one-hot label output by the classification network into The noise samples in the two domains are treated as target labels with noise corruption, and label propagation is used to purify the noise samples in the two domains. Label propagation can smooth the labels of adjacent samples so that these samples tend to have the same labels, thereby extracting robust class information from the two domains for more reliable supervision.

[0064] Based on the above ideas, an embodiment of the present invention proposes to use label propagation to remove noise samples. Specifically, a graph structure is established based on all samples in the source domain data and the target domain data. Label propagation is performed on the noise samples based on the established graph structure to purify the noise samples and obtain robust label matrices corresponding to the source domain data and the target domain data, respectively, including:

[0065] Based on the similarity between all samples in the source and target domains, a corresponding graph structure is established. The embedding distance between adjacent nodes in the graph structure is calculated to construct the corresponding adjacency matrix. Based on the adjacency matrix and the initial hard label matrix corresponding to the clean samples in the source and target domains, label propagation is iterated to obtain the soft label matrix corresponding to the noise samples. The robust label matrix is ​​composed of the initial hard label matrix and the soft label matrix. More specifically:

[0066] The corresponding graph structure is established based on the similarity between all samples in the target domain data and the source domain data. The similarity calculation refers to the existing technology. Here, taking the target domain data as an example, the embedding distance between adjacent nodes in the corresponding graph structure of the target domain data is first calculated to obtain the adjacency matrix The embedding distance calculation formula between adjacent nodes in the adjacency matrix A is expressed as:

[0067]

[0068] in, represent the i-th sample and the j-th sample respectively, Representation sample The adjacent set of Represents samples sample The feature extraction function of . Then the adjacency matrix A is passed Normalized, the matrix D is a diagonal matrix, the initial hard label matrix in the target domain data in,

[0069]

[0070] in, represents the i-th sample The initial hard label matrix of the sample is a clean sample in the target domain data, One-hot(·) means mapping the label to a One-hot vector, Then, in the initial hard label matrix Y t(0) Label propagation is performed on the basis of K iterations (k = 0, 1, ..., Kl), based on Perform label propagation. The label propagation formula in the kth label propagation iteration is expressed as:

[0071]

[0072] in, is the label matrix of all samples in the target domain after the (k-1)th label propagation, represents the i-th sample The soft labels after the (k-1)th label propagation are used to update the soft labels of the noise samples with more reliable class information. After the label propagation, the smoothing constraint and fitting constraint of the regularized label propagation process are applied. The smoothing constraint requires that the soft labels of adjacent samples are as similar as possible, while the fitting constraint requires that the result does not change much from the initial label assignment. The trade-off between these two competing constraints is captured by the trade-off parameter γ (γ>0). Finally, the soft label matrix corresponding to the noise samples in the target domain data is expressed as:

[0073]

[0074] Among them, Y t* represents the soft label matrix corresponding to the noise sample in the target domain data, I represents the identity matrix, γ represents the trade-off parameter, γ>0, represents the degree matrix of the adjacency matrix A, D represents a diagonal matrix, and the adjacency matrix A is the element A ij A collection of ij represents the embedding distance between the i-th sample and the j-th sample, represent the i-th sample and the j-th sample respectively, Representation sample The adjacent set of Represents samples sample The feature extraction function, Y t(0) represents the initial hard label matrix, represents the i-th sample The initial hard label matrix, One-hot(·) represents the pseudo-label one-hot vector corresponding to the classification network output. Represents the set of clean samples in the target domain data.

[0075] Thus, the robust class information in the target domain data can be obtained, that is, the soft label matrix Y t* The soft label and initial hard label matrix Y of the noise sample t(0) The hard label composition of the clean sample in is recorded as This is the robust label matrix corresponding to the target domain data. The label propagation operation in the source domain data is the same as that in the target domain data, so we will not go into details here. Finally, the soft label matrix The soft label and initial hard label matrix Y of the noise sample s(0) The hard label composition of the clean sample in is recorded as That is, the robust label matrix corresponding to the target domain data. During the entire label propagation process, the initial hard label matrix Y of the clean samples in the target domain data and the source domain data is t(0) 、Y s(0) No change, it is only used here as label initialization for label propagation.

[0076] Therefore, the objective loss function formula constructed based on the robust label matrix proposed in the embodiment of the present invention is expressed as:

[0077] l robust =l sup +η·l d (D(G(X s )), D(G(X t ))),

[0078] Among them, l robust represents the target loss function, l ce (·,·) is the cross entropy loss, L(G(X s )) is to extract feature X from the source domain data S The predicted label of for The transposed vector form of Represents the initial hard label matrix Y corresponding to the source domain data s(0) and the soft label matrix The robust label matrix composed of L(G(X t )) is to extract feature X from the target domain data t The predicted label of for The transposed vector form of Represents the initial hard label matrix Y corresponding to the target domain data t(0)and the soft label matrix Y t* The robust label matrix composed of η represents the contribution parameter, l d (·,·) is the domain similarity loss, D(G(X S )) is to extract feature X from the source domain data S The domain where D(G(X t )) is to extract feature X from the target domain data t The domain where the .

[0079] It can be seen that when the potential robust class information of the two domains is mined, the classification network can be updated by optimizing the target loss function, thereby using the updated and better classification network to obtain better class discrimination and robustness. Figure 4 ,like Figure 4 The whole robust class information extraction process for these two domains is illustrated. Figure 4 The classifier includes a domain classifier and a label predictor. During the optimization process, robust class information extraction is performed every M iterations, that is, the robust class supervision information in the two domains is updated every M iterations.

[0080] S40. On the selected clean samples, a cross-domain denoising contrastive learning method including cross-domain intra-class alignment and inter-class dispersion is respectively performed; wherein, a distribution matching function is constructed based on the cross-domain intra-class alignment, and an embedding discrete function is constructed based on the inter-class dispersion.

[0081] Although the S30 process constrains alignment at the domain level and provides robust class supervision information in both domains, it still cannot guarantee class distinguishability. In other words, it lacks constraints that can push samples of different categories apart, resulting in blurred classification boundaries. To this end, the present embodiment proposes a cross-domain denoising contrastive learning method on selected clean samples, including cross-domain intra-class alignment and inter-class dispersion, to improve the model's class-level distinguishability and enhance the representation learning process.

[0082] For cross-domain intra-class alignment, in order to combat label noise, the clean sample embedding prototype alignment target is used. The present invention provides an optional solution, and the process of performing cross-domain intra-class alignment includes:

[0083] The clean samples in the source domain data and the target domain data are used as embedding prototypes to calculate the cross-domain intra-class distribution matching and achieve cross-domain intra-class alignment. The distribution matching function formula constructed based on cross-domain intra-class alignment is expressed as:

[0084]

[0085] Among them, l dm represents the distribution matching function, C represents the number of classes of source domain data or target domain data, φ(·, ·) represents the calculation of the square of the Euclidean distance, represents the embedded prototype of the k-th clean sample in the source domain data, Describes the embedding prototype of the kth class of clean samples in the target domain data. The embedding prototype of the kth class is calculated by the mean of the embeddings of the clean samples belonging to the kth class. During training, the embedding prototype of the kth class is initialized with the mean of all clean samples belonging to the kth class. In each iteration, if there are no clean samples of the kth class in the batch, the embedding prototype obtained in the previous iteration is considered the current embedding prototype.

[0086] For inter-class dispersion, contrastive learning constraints are used to represent the distinction between different classes in the two domains. The present invention provides an optional solution, and the process of performing inter-class dispersion includes:

[0087] The embedding discretization calculation in the two domains is performed by taking the clean samples in the source domain data as the object, and the embedding prototype of the sample of the same category in the target domain data corresponding to the clean sample as the positive sample, and the clean sample of a different category from the clean sample as the negative sample to achieve inter-class dispersion. The embedding discretization function formula constructed based on inter-class dispersion is expressed as:

[0088]

[0089] Among them, l cl represents the embedded discrete function, represents the number of clean samples in the source domain data, Represents a set of clean samples in the source domain data, sim(·,·) represents the similarity evaluation function for two items, and the similarity is obtained based on the square of the Euclidean distance. represents the label of the i-th clean sample in the source domain data, Indicates the corresponding label in the target domain data The embedding prototype of samples of the same category, i.e. As a positive sample, Represent the negative samples in the source domain data Negative samples The feature extraction function.

[0090] Here, only clean samples from the source domain data of different classes are used as negative samples for two reasons. First, clean samples in the source domain data have higher confidence than clean samples in the target domain data. Second, since cross-domain intra-class alignment has already achieved class alignment between the two domains, there is no need to use clean samples in the target domain data as negative samples again in cross-domain contrastive learning. In addition, positive samples are usually assumed to belong to the same class as the reference samples, but during the training process, there may be only one clean sample in a batch of a class. Therefore, to avoid the phenomenon of zero positive samples, the embodiments of the present invention use target prototypes of the same class as the positive samples.

[0091] S50, constructing a final target optimization function according to the target loss function, the distribution matching function and the embedding discrete function.

[0092] Based on the considerations of S30 and S40 above, the embodiment of the present invention proposes that the final target optimization function constructed according to the target loss function, the distribution matching function and the embedded discrete function is expressed as follows:

[0093] l=l robust +l dm +l cl ,

[0094] Among them, l represents the final target optimization function, l robust represents the target loss function, l dm represents the distribution matching function, l cl Represents an embedded discrete function. robust For the proposed cross-domain robust class information constraint, l dm and l cl Cross-domain contrastive learning constraints for the proposed denoising.

[0095] S60. Classify the target domain data according to the classification network and the target optimization function iteration.

[0096] Based on the classification network selected in S20 and the final target optimization function constructed based on S30 and S40, the network parameters of the classification network are iteratively updated. The classification network outputs the classification accuracy of different classes in each domain. The iterative process continues until the best classification result is output.

[0097] In order to verify the effectiveness of the robust cross-domain adaptive classification method based on denoising contrastive learning provided by the embodiment of the present invention, the following experiments are conducted for verification.

[0098] Visualization using t-SNE algorithm, Figure 5 (a)~ Figure 5 (d) shows the classification capabilities of the embedded space for the original data, the Domain Adaptive Neural Networks (DANN), the Robust Domain Adaptation (RDA), and the proposed method for A→W in the Office-31 dataset. The different colors in the figure represent different types of data. As can be seen in 5(a), the original data is scattered and unclassified; as can be seen in 5(b), the DANN method fails to classify the data well and is also scattered; as can be seen in 5(c), many samples in the RDA results are completely misclassified, indicating obvious class confusion; as can be seen in 5(d), the proposed method performs very well in comparison.

[0099] In summary, the robust cross-domain adaptive classification method based on denoising contrastive learning proposed in the embodiment of the present invention can simultaneously generate robust discrimination of noise samples and domain-invariant representations, specifically: obtaining source domain data and target domain data; wherein the source domain data and the target domain data share the same label space; the source domain data and the target domain data respectively include clean samples and noise samples, and the source domain data are all labeled samples, and the target domain data are all unlabeled samples; using the classification network to select clean samples from the source domain data and the target domain data respectively; establishing a graph structure on all samples in the source domain data and the target domain data, and according to the established A stand-alone graph structure is used to propagate labels on noise samples to purify the noise samples and obtain the robust label matrices corresponding to the source domain data and the target domain data respectively, and the target loss function is constructed according to the robust label matrix; on the selected clean samples, cross-domain denoising contrastive learning methods including cross-domain intra-class alignment and inter-class dispersion are respectively performed; among them, the distribution matching function is constructed based on cross-domain intra-class alignment, and the embedding discrete function is constructed based on inter-class dispersion; the final target optimization function is constructed according to the target loss function, distribution matching function and embedding discrete function; the classification network and the target optimization function are iteratively implemented to achieve the classification of the target domain data. It can be seen that in addition to considering the class information of the source domain data, the embodiments of the present invention also consider extracting robust class information from the target domain data to provide more effective supervision; the embodiments of the present invention simultaneously select clean samples for the source domain data and the target domain, and filter out noise samples in the two domains through label propagation, instead of directly discarding the noise samples, thereby obtaining sufficient robust class information in the two domains, extracting robust class information to supervise the network to learn class-level discriminant representation, making supervision more reliable, thereby enhancing the class-level distinction ability; the embodiments of the present invention also propose a cross-domain denoising contrast learning method, which operates on the clean samples selected in the two domains, so as to better ensure cross-domain class invariance, improve the class-level distinction ability of the model, enhance the robustness of the representation learning process, and produce more robust classification results.

[0100] See Figure 6 , an embodiment of the present invention provides an electronic device, including a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other via the communication bus 604;

[0101] Memory 603, used for storing computer programs;

[0102] The processor 601 is configured to implement the steps of the above-mentioned robust cross-domain adaptive classification method based on denoising contrastive learning when executing the program stored in the memory 603.

[0103] An embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned robust cross-domain adaptive classification method based on denoising contrastive learning.

[0104] As for the device / electronic device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0105] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0106] Although the present invention is described herein in conjunction with various embodiments, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the specification and accompanying drawings in the process of implementing the claimed invention. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components or steps. The fact that certain measures are described in different embodiments does not mean that these measures cannot be combined to produce good results.

[0107] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A robust cross-domain adaptive classification method based on denoising contrastive learning, applied to image classification tasks, characterized by: include: Acquire source domain data and target domain data; wherein the source domain data and the target domain data share the same label space; the source domain data and the target domain data are both image data; the source domain data and the target domain data respectively include clean samples and noisy samples, and the source domain data are all labeled samples, and the target domain data are all unlabeled samples; Selecting clean samples from the source domain data and the target domain data respectively using a classification network; Establishing a graph structure based on all samples in the source domain data and the target domain data, performing label propagation on noise samples based on the established graph structure to purify the noise samples to obtain robust label matrices corresponding to the source domain data and the target domain data, respectively, and constructing a target loss function based on the robust label matrices; On the selected clean samples, a cross-domain denoising contrastive learning method, including cross-domain intra-class alignment and inter-class dispersion, is performed. A distribution matching function is constructed based on cross-domain intra-class alignment, and an embedding discrete function is constructed based on inter-class dispersion. Constructing a final target optimization function according to the target loss function, the distribution matching function and the embedded discrete function; The classification of the target domain data is achieved iteratively according to the classification network and the target optimization function.

2. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 1 is characterized in that Selecting clean samples from the source domain data and the target domain data using a classification network, including: Outputting cross entropy loss values ​​corresponding to all samples in the source domain data and the target domain data using a classification network; Compare the cross entropy loss value with the confidence threshold. If the cross entropy loss value is less than or equal to the confidence threshold, select the sample corresponding to the cross entropy loss value as a clean sample; otherwise, select the sample corresponding to the cross entropy loss value as a noise sample.

3. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 1, characterized in that Establishing a graph structure based on all samples in the source domain data and the target domain data, and performing label propagation on noise samples according to the established graph structure to purify the noise samples to obtain robust label matrices corresponding to the source domain data and the target domain data, respectively, including: Establishing corresponding graph structures according to the similarities between all samples in the source domain data and the target domain data; Calculating the embedding distances between adjacent nodes in the graph structure to construct a corresponding adjacency matrix; Performing label propagation iteration processing according to the adjacency matrix and the initial hard label matrices corresponding to the clean samples in the source domain data and the target domain data to obtain a soft label matrix corresponding to the noise sample; The initial hard label matrix and the soft label matrix form a corresponding robust label matrix.

4. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 3 is characterized in that The soft label matrix formula corresponding to the noise sample in the target domain data is expressed as: Among them, Y t* represents the soft label matrix corresponding to the noise samples in the target domain data, I represents the unit matrix, γ represents the trade-off parameter, γ>0, represents the degree matrix of the adjacency matrix A, D represents a diagonal matrix, and the adjacency matrix A is the element A ij A collection of ij represents the embedding distance between the i-th sample and the j-th sample, represent the i-th sample and the j-th sample respectively, Representation sample The adjacent set of Represents samples sample The feature extraction function, Y t(0) represents the initial hard label matrix, represents the i-th sample The initial hard label matrix, One-hot(·) represents the pseudo-label one-hot vector corresponding to the classification network output. Represents the set of clean samples in the target domain data.

5. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 4 is characterized in that The objective loss function formula constructed according to the robust label matrix is ​​expressed as: l robust =l sup +η·l d (D(G(X s )),D(G(X t ))), Among them, l robust represents the target loss function, l ce (·,·) is the cross entropy loss, L(G(X s )) is to extract feature X from the source domain data S The predicted label of for The transposed vector form of Represents the initial hard label matrix Y corresponding to the source domain data s(0) and the soft label matrix Y s* The robust label matrix composed of L(G(X t )) is to extract feature X from the target domain data t The predicted label of for The transposed vector form of Represents the initial hard label matrix Y corresponding to the target domain data t(0) and the soft label matrix Y t* The robust label matrix composed of η represents the contribution parameter, l d (·,·) is the domain similarity loss, D(G(X S )) is to extract feature X from the source domain data S The domain where D(G(X t )) is to extract feature X from the target domain data t The domain where the .

6. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 5, characterized in that The process of performing cross-domain intra-class alignment includes: Clean samples in the source domain data and the target domain data are used as embedding prototypes to calculate cross-domain intra-class distribution matching, thereby achieving cross-domain intra-class alignment.

7. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 6, characterized in that: The distribution matching function formula based on cross-domain intra-class alignment is expressed as: Among them, l dm represents the distribution matching function, C represents the number of classes of source domain data or target domain data, φ(·, ·) represents the calculation of the square of the Euclidean distance, represents the embedded prototype of the k-th clean sample in the source domain data, Describe the embedding prototype of the k-th clean sample in the target domain data.

8. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 7, characterized in that: The process of performing inter-class dispersion includes: Taking the clean samples in the source domain data as the object, the embedded prototype of the sample of the same category in the target domain data corresponding to the clean sample as the positive sample, and the clean sample of a different category from the clean sample as the negative sample, the discrete embedding calculations in the two domains are performed to achieve inter-class dispersion.

9. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 8, characterized in that: The embedding discrete function formula based on inter-class dispersion is expressed as: Among them, l cl represents the embedded discrete function, represents the number of clean samples in the source domain data, represents the set of clean samples in the source domain data, sim(·,·) represents the similarity evaluation function for two items, represents the label of the i-th clean sample in the source domain data, Indicates the corresponding label in the target domain data The embedding prototype of samples of the same category, i.e. As a positive sample, Represent the negative samples in the source domain data Negative samples The feature extraction function.

10. The robust cross-domain adaptive classification method based on denoising contrastive learning according to claim 9, characterized in that: The final objective optimization function formula constructed according to the objective loss function, the distribution matching function and the embedded discrete function is expressed as: l＝l robust +l dm +l cl ， Among them, l represents the final target optimization function, l robust represents the target loss function, l dm represents the distribution matching function, l cl represents an embedded discrete function.

Images