Feature selection method and system based on domain adaptation and domain adversarial training

Abstract

The invention is suitable for the technical field of machine learning, and provides a feature selection method and system based on domain adaptation and domain adversarial training, and the method comprises the following steps: obtaining source domain data and target domain data, constructing a source domain feature selection target function, and generating a binary feature mask vector; constructing a deep transfer learning framework based on a domain adversarial neural network, training the domain adaptive neural network by using the source domain tagged data and the target domain untagged data in a domain adversarial form, and constructing a cross-domain shared feature representation space of the source domain and the target domain; and feature selection knowledge migration from the source domain to the target domain is realized through decoding conversion. According to the method, the feature distribution difference between the source domain and the target domain is effectively eliminated through the adversarial training strategy driven by the gradient inversion layer, the method has remarkable advantages in a target domain data scarcity scene, the data annotation cost can be reduced, cross-domain potential association can be captured, and redundant features and over-fitting risks are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning technology, and in particular relates to a feature selection method and system based on domain adaptation and domain adversarial training. Background Art

[0002] Feature selection is a key process in machine learning, which aims to improve model performance by identifying the most relevant features from the dataset, thereby reducing dimensionality and improving interpretability. Traditional feature selection methods usually require a large amount of labeled data for effective feature evaluation and selection. As a key preprocessing step in high-dimensional data processing, its core value lies in eliminating irrelevant features and redundant dimensions to construct an optimal feature subset to improve model generalization ability, reduce computational complexity and enhance decision interpretability. Especially in scenarios such as multi-label classification, due to the complex nonlinear coupling relationship between samples and multiple labels, traditional feature selection methods face two challenges: first, statistical evaluation methods based on the label independence assumption (such as chi-square test and mutual information method) are difficult to capture the potential correlation between labels; second, limited by the paradigm constraints of supervised learning, when labeled data in the target domain is scarce, the reliability of feature importance evaluation is significantly reduced.

[0003] In many real-world applications, the target task often has limited or incomplete data, making efficient feature selection difficult. Transfer learning, however, allows knowledge transfer from other tasks to be used to perform feature selection in the target task using a small amount of labeled data. This is particularly important in scenarios where data acquisition and annotation are expensive, such as medical image analysis and text classification.

[0004] The intersection of feature selection and transfer learning represents a promising frontier in machine learning research. By leveraging existing knowledge from related fields, transfer learning facilitates more accurate and computationally efficient feature selection processes, enabling the development of robust and scalable machine learning solutions for complex multi-label tasks. As research in this area continues to advance, these techniques are expected to play an increasingly important role in creating sophisticated models capable of handling the challenges posed by multi-label data. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a feature selection method based on domain adaptation and domain adversarial training, aiming to solve the problems raised in the above background technology.

[0006] The embodiment of the present invention is implemented as follows: a feature selection method based on domain adaptation and domain adversarial training includes the following steps:

[0007] Obtain source domain data and target domain data, construct the source domain feature selection objective function, and generate a binary feature mask vector;

[0008] Build a deep transfer learning framework based on domain adversarial neural networks. Use labeled data from the source domain and unlabeled data from the target domain to train a domain adaptation neural network, and construct a feature representation space shared across the source and target domains.

[0009] The feature selection knowledge transfer from the source domain to the target domain is achieved through decoding conversion, and the feature selection results of the source domain are converted into the feature selection results of the target domain through the decoder.

[0010] Another object of an embodiment of the present invention is to provide a feature selection system based on domain adaptation and domain adversarial training, which is used in the above-mentioned feature selection method, including:

[0011] The data acquisition module is used to obtain source domain data and target domain data, construct the source domain feature selection objective function, and generate a binary feature mask vector;

[0012] The transfer learning framework construction module is used to build a deep transfer learning framework based on domain adversarial neural networks. It trains the domain adaptation neural network using labeled data from the source domain and unlabeled data from the target domain through domain adversarial methods, and constructs a feature representation space shared across the source and target domains.

[0013] The feature selection knowledge transfer module is used to realize the feature selection knowledge transfer from the source domain to the target domain through decoding conversion, and converts the source domain feature selection results into the target domain feature selection results through the decoder.

[0014] To overcome the challenges posed by the scarcity of labeled data, the embodiments of the present invention utilize transfer learning. Transfer learning leverages knowledge from related domains or tasks (referred to as source domains) to improve learning performance in target domains with limited data or labels. By promoting knowledge reuse, transfer learning can provide additional insights and guidance for the feature selection process by adjusting the distribution between domains. It leverages knowledge from the source domain to improve learning in the target domain, guiding feature selection even under data-scarce conditions. The synergy between transfer learning and feature selection opens up new avenues for enhancing feature selection methods and expanding their applicability.

[0015] The embodiments of the present invention deeply combine domain adaptation technology with feature selection methods to ensure effective knowledge transfer. This combination provides a new solution to the feature selection problem, breaking through the limitations of existing methods that are difficult to cope with scenarios where data or labeled data is scarce. The framework proposed in the embodiments of the present invention is designed to be highly compatible with existing feature selection methods. It can be successfully integrated with popular methods such as mutual information-based filters, recursive feature elimination, and Lasso regression to improve their performance in data-scarce scenarios without requiring large-scale modifications.

[0016] The embodiment of the present invention provides a novel feature selection method by integrating the domain adaptation transfer learning method with the feature selection method, which can realize feature selection tasks in scenarios such as scarce data or missing data labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a feature selection method based on domain adaptation and domain adversarial training provided by an embodiment of the present invention;

[0018] Figure 2 Flowchart of a feature selection method based on domain adaptation and adversarial training provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] In response to the problem of feature selection effect degradation caused by scarcity of target domain data in cross-domain scenarios, an embodiment of the present invention provides a feature selection method based on domain adaptation and adversarial training, which is used to explore the shared feature subspace of source domain data and target domain, thereby providing a possibility to solve scenarios such as scarce data or missing data labels. Through the cross-domain knowledge transfer mechanism, the feature representation paradigm or selection criteria learned in the source domain is effectively transferred to the feature optimization process of the target domain; through adversarial training, a domain-invariant feature space is constructed, so that the model can learn the shared feature space of the source domain and the target domain; a feature reconstruction decoding conversion structure is introduced to ensure the stability of the feature selection criterion in cross-domain migration; the domain discriminator design based on the gradient reversal layer effectively eliminates the feature distribution offset between domains.

[0021] The specific implementation of the present invention is described in detail below with reference to specific embodiments.

[0022] like Figure 1 FIG. 1 is a flowchart of a feature selection method based on domain adaptation and domain adversarial training provided by an embodiment of the present invention, comprising the following steps:

[0023] S1. Obtain source domain data and target domain data, construct the source domain feature selection objective function, and generate a binary feature mask vector:

[0024] Get the source domain data D (sufficient and well-labeled data, including m samples, consisting of samples with the same feature space but different labels in the source domain) s , feature space A set of labels {y1,y2,...,y k}, the source domain data is defined as Ds ={x i1 ,x i2 ,...x im ,y i} {m} {i＝1} ( );

[0025] The target domain (data is sparse or annotations are missing, containing n samples, consisting of samples with the same feature space but different labels in the target domain) is defined as D t ={z j1 , z j2 ,...,z jn} {n} {j＝1} ( ), feature space Source domain feature space and the target domain feature space Having the same dimensions but with significantly different distributions;

[0026] Construct source domain feature selection objective function F s :X s →{0, 1} d , generating a binary feature mask vector.

[0027] S2. Build a deep transfer learning framework based on domain adversarial neural networks. Use labeled data from the source domain and unlabeled data from the target domain to train a domain adaptation neural network, and build a feature representation space shared across the source and target domains.

[0028] A deep transfer learning framework based on a Domain-Adversarial Neural Network (DANN) is constructed. The goal of the DANN is to make the distribution of data in the source and target domains as consistent as possible by learning a new shared feature space. Therefore, after passing through the feature extraction module of the DANN, the feature spaces of the source and target domains will change to achieve domain alignment. The main structure includes a feature dimension adapter, feature extractor, label predictor, domain discriminator, and decoder.

[0029] The overall training process is as follows Figure 2As shown, a training data set is composed based on data information in related fields. The training data set is used as input. Based on the feature dimension adapter, the source domain and target domain data of different dimensions are aligned to the same dimension. Based on the feature extractor, a feature space shared by the source domain and target domain data is created. Based on the label predictor, the source domain data is classified. Based on the domain discriminator, it is determined whether the input data comes from the source domain space or the target domain space. A gradient reversal layer is introduced to achieve domain alignment. Based on the decoder, the feature vector of the candidate feature subset is obtained.

[0030] For the feature extractor, sigmoid is used as the activation function and the output is: G f (x; W, b)) = sigmoid(Wx+b), where G f is the feature extraction function, which represents the feature map of the input data after passing through the feature extractor, x is the input sample data vector, W is the weight matrix, and b is the bias term;

[0031] For the label predictor, softmax is used as the activation function and the output is: G y (G f (x); V, c) = softmax(VG f (x)+c), where G y is the label prediction function, outputting the category probability distribution, V is the classification weight matrix, and c is the classification bias term;

[0032] When a data point (x i ,y i ), with negative log-likelihood as the loss function, the loss of the label predictor is: in, represents the negative log-likelihood loss for classification tasks;

[0033] For the domain discriminator, sigmoid is used as the activation function, and its output is: Among them, G d represents the domain discriminant function, which is used to determine whether the sample belongs to the source domain or the target domain, u is the domain discriminant weight vector, and z is the domain discriminant bias term;

[0034] Negative log likelihood is used as loss function to define domain discriminator G d The loss of (·) is: Among them, d i Indicates the domain label, d i =1 indicates source domain samples, d i =0 indicates the target domain sample;

[0035] Define the loss function Loss total Losstotal =Loss label -lambda*Loss domain , among which, Loss label is the label predictor loss, Loss domain is the domain discriminator loss, lambda is the adaptation parameter of the gradient reversal layer;

[0036] The constructed deep transfer learning framework based on domain adversarial neural network is trained using domain adversarial training to enable the model to learn domain invariant features.

[0037] The core idea of feature space alignment is that the feature extractor not only learns the features of the classification task, but also, under the constraints of adversarial learning, makes the features of the source and target domains indistinguishable by the domain classifier in the shared space. After training, the features of the source and target domains are remapped in the shared feature space learned by the domain adversarial neural network, making their distribution more consistent.

[0038] S3. Transfer feature selection knowledge from the source domain to the target domain through decoding conversion. The feature selection results of the source domain are converted into the feature selection results of the target domain through the decoder:

[0039] The feature selection results of the source domain are used as supervisory signals to guide the generation of shared spatial features;

[0040] The decoder is used to map the feature subset of the shared space to the target domain, and the most similar candidate feature subset is calculated as the result of target domain feature selection.

[0041] Finally, the data in the test data set can also be used to test the trained model to verify whether the target domain feature selection results improve the performance of the classification results. Specifically, the candidate subsets obtained by directly using the feature selection method adopted in the source domain in the target domain data set and the candidate subsets obtained by the embodiment of the present invention are selected respectively, and the classification is verified by testing different feature subsets.

[0042] When the data of the target task is sparse or incomplete, existing technologies find it difficult to support efficient feature selection. In order to overcome the challenges brought about by the scarcity of labeled data, the embodiments of the present invention utilize knowledge of related fields or tasks (referred to as source fields) to improve the learning performance of target fields with limited data or labels. By promoting knowledge reuse, additional insights and guidance can be provided to the feature selection process by adjusting the distribution between fields, and the knowledge of the source field can be used to improve the learning of the target field, thereby guiding feature selection even under conditions of data scarcity.

[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A feature selection method based on domain adaptation and domain adversarial training, characterized in that: The following steps are involved: Obtain source domain data and target domain data, construct the source domain feature selection objective function, and generate a binary feature mask vector; Build a deep transfer learning framework based on domain adversarial neural networks. Use labeled data from the source domain and unlabeled data from the target domain to train a domain adaptation neural network, and construct a feature representation space shared across the source and target domains. The feature selection knowledge transfer from the source domain to the target domain is achieved through decoding conversion, and the feature selection results of the source domain are converted into the feature selection results of the target domain through the decoder.

2. The feature selection method based on domain adaptation and domain adversarial training according to claim 1, characterized in that: The steps of obtaining source domain data and target domain data, constructing a source domain feature selection objective function, and generating a binary feature mask vector specifically include: Get source domain data D s , feature space A set of labels {y1, y2, ..., y k }, the source domain data is defined as D s ={x i1 , x i2 ,...x im ,y i } {m} {i＝1} ; The target domain data is defined as D t ={z j1 , z j2 ,...,z jn } {n} {j＝1} Feature Space Construct source domain feature selection objective function F s :X s →{0, 1} d , generating a binary feature mask vector.

3. The feature selection method based on domain adaptation and domain adversarial training according to claim 2, characterized in that: The steps of constructing a deep transfer learning framework based on a domain adversarial neural network, training a domain adaptation neural network with source domain labeled data and target domain unlabeled data in the form of domain adversarial training, and constructing a feature representation space shared across source and target domains specifically include: Build a deep transfer learning framework based on domain adversarial neural networks, including feature dimension adapters, feature extractors, label predictors, domain discriminators, and decoders; A training dataset is formed based on data information in related fields. The training dataset is used as input. The source and target domain data of different dimensions are aligned to the same dimension based on the feature dimension adapter. A feature space shared by the source and target domain data is created based on the feature extractor. The source domain data is classified based on the label predictor. The domain discriminator determines whether the input data comes from the source domain space or the target domain space. The decoder is used to obtain the feature vectors of the candidate feature subset. The constructed deep transfer learning framework based on domain adversarial neural network is trained using domain adversarial training.

4. The feature selection method based on domain adaptation and domain adversarial training according to claim 3, characterized in that: For the feature extractor, sigmoid is used as the activation function and the output is: G f (x; W, b) = sigmoid(Wx+b), where G f is the feature extraction function, which represents the feature map of the input data after passing through the feature extractor, x is the input sample data vector, W is the weight matrix, and b is the bias term; For the label predictor, softmax is used as the activation function and the output is: G y (G f (x); V, c) = softmax(VG f (x)+c), where G y is the label prediction function, outputting the category probability distribution, V is the classification weight matrix, and c is the classification bias term; When a data point (x i ,y i ), with negative log-likelihood as the loss function, the loss of the label predictor is: in, represents the negative log-likelihood loss for classification tasks; For the domain discriminator, sigmoid is used as the activation function, and its output is: Among them, G d represents the domain discriminant function, which is used to determine whether the sample belongs to the source domain or the target domain, u is the domain discriminant weight vector, and z is the domain discriminant bias term; Negative log likelihood is used as loss function to define domain discriminator G d The loss of (·) is: Among them, d i Indicates the domain label, d i =1 indicates source domain samples, d i =0 indicates the target domain sample; Define the loss function Loss total Loss total =Loss label -lambda*Loss domain , among which, Loss label is the label predictor loss, Loss domain is the domain discriminator loss, and lambda is the adaptation parameter of the gradient reversal layer.

5. The feature selection method based on domain adaptation and domain adversarial training according to claim 3, characterized in that: The step of achieving feature selection knowledge transfer from the source domain to the target domain through decoding conversion, and converting the source domain feature selection result into the target domain feature selection result through the decoder, specifically includes: The feature selection results of the source domain are used as supervisory signals to guide the generation of shared spatial features; The decoder is used to map the feature subset of the shared space to the target domain, and the most similar candidate feature subset is calculated as the result of target domain feature selection.

6. A feature selection system based on domain adaptation and domain adversarial training, used to implement the feature selection method based on domain adaptation and domain adversarial training according to any one of claims 1 to 5, characterized in that: include: The data acquisition module is used to obtain source domain data and target domain data, construct the source domain feature selection objective function, and generate a binary feature mask vector; The transfer learning framework construction module is used to build a deep transfer learning framework based on domain adversarial neural networks. It trains the domain adaptation neural network using labeled data from the source domain and unlabeled data from the target domain through domain adversarial methods, and constructs a feature representation space shared across the source and target domains. The feature selection knowledge transfer module is used to realize the feature selection knowledge transfer from the source domain to the target domain through decoding conversion, and converts the source domain feature selection results into the target domain feature selection results through the decoder.

Images