A Source Domain Selection Method Based on Metric Learning

Abstract

The present invention relates to a source domain selection method based on metric learning. By aligning the electroencephalogram (EEG) signals of the source domain and the target domain in the Euclidean space, the data distributions among different subjects become more similar. Considering that the dimension of EEG signals is usually relatively high, the linear discriminant method is used to reduce the dimension of the aligned data, while reducing the within-class difference of samples and increasing the between-class difference of samples. Finally, a metric function is designed to calculate the within-class distance and between-class distance of samples, and this is used as a criterion to measure the similarity between the source domain and the target domain, so as to select the most suitable source domain for migration.

Description

Technical Field

[0001] The present invention relates to a source domain selection method based on metric learning, belonging to the technical field of pattern recognition. It is a method that performs Euclidean alignment on the original electroencephalogram (EEG) data and uses the metric learning method to calculate the similarity between the source domain and the target domain to select the optimal source domain. Background Art

[0002] A brain-computer interface (BCI) is a direct communication pathway established between the brain and the outside world without relying on the participation of peripheral nerves and muscle tissues, and is an important means to achieve human-computer interaction. And the classification of electroencephalogram (EEG) signals based on motor imagery is a typical paradigm in BCI technology. It uses an EEG acquisition device to obtain EEG signals in the motor imagery area and uses machine learning algorithms for discrimination, and converts the results into control commands of the device to complete the corresponding imagined actions. For patients with disorders of consciousness and stroke, it can assist in the rehabilitation of patients. At the same time, patients can control mechanical equipment through this technology to complete daily actions, making the lives of patients more convenient.

[0003] In essence, a BCI system can be regarded as a pattern recognition system, and its most fundamental purpose is to obtain a high accuracy rate for external device control, and the signal recognition effect directly affects the recognition performance of the system. Common learning algorithms include support vector machines, Bayesian classifiers, hidden Markov models, neural networks, and combined classifiers, etc. However, these algorithms essentially belong to supervised learning. If a supervised learning algorithm wants to obtain an offline supervised classifier with a high recognition rate, it requires a large amount of training and calibration of the subject to obtain labeled samples that are sufficient to represent and cover the EEG signals of the subject. However, in the actual application of the BCI system, it is very difficult to obtain sufficient and representative samples that can cover the entire data space. And due to the time-varying nature of EEG signals themselves, as well as the differences between individuals and within individuals, it is difficult to establish a general algorithm model applicable to all subjects. Many research teams have successively studied the theory and methods of transfer learning, migrating the labeled data or models of source subjects to target subjects, reducing or even eliminating the calibration time of target subjects, and improving the adaptability of the algorithm model. The idea of transfer learning is to apply the knowledge or patterns learned in a certain field or task to different but related fields or problems. Since a basic assumption of traditional machine learning is that the training and test data come from the same distribution. However, this assumption often cannot be satisfied in practical applications. Therefore, when the transfer of data or knowledge from the source domain has a negative impact on the learning of the target domain, negative transfer will occur. Therefore, when the data quality of the source domain and the target domain is fixed, negative transfer can be effectively avoided by selecting the source domain with the smallest domain difference for migration. Currently, a relatively popular method to solve this problem is metric learning.

[0004] The basic idea of metric learning is to construct a metric learning function with the prior knowledge of some pre-observed samples given a part of samples, and then learn an optimal metric between these samples and satisfy the pre-given constraints.

[0005] To select the most suitable source domain samples of motor imagery for transfer, the present invention proposes a source domain selection method based on metric learning, which combines Euclidean alignment and linear discriminant analysis to construct a metric learning function for measuring the similarity between domains. Summary of the Invention

[0006] The object of the present invention is to provide a source domain selection method based on metric learning for the deficiencies of the prior art.

[0007] To achieve the above object, the technical solution of the present invention is:

[0008] A source domain selection method based on metric learning, comprising the following steps:

[0009] Step 1, perform Euclidean alignment on the original EEG data;

[0010] Assume that the data of the subjects in each source domain is D = {x i , y i}, y i ∈ {0, 1}, i = 1, 2,..., n, where n is the number of times the same subject is tested, x i represents the i-th sample, and y i refers to the label of the corresponding sample. Then, without the labels of the source domain and the target domain, the following reference matrix can be calculated:

[0011]

[0012] Where is the arithmetic mean of all covariance matrices from one subject. The data after Euclidean alignment is denoted as:

[0013]

[0014] Step 2, perform linear discriminant analysis using the data after Euclidean alignment;

[0015] The specific implementation is as follows: Assume that the data after Euclidean alignment is n is the number of times the same subject is tested, represents the i-th sample, and y i refers to the label of the corresponding sample. Let μ i , Σ iDenote the set, mean, and covariance matrix of the \(i\)-th class of samples respectively. Denote the projection space as \(w\), then the projections of the centers of the two classes of samples in the space are \(w\). T \(\mu_0\) and \(w\) T \(\mu_1\); if all sample points are projected onto the projection space, then the covariances of the two classes of samples are \(w\) T \(\Sigma_0 w\) and \(w\) T \(\Sigma_1 w\). Among them, \(w\) T \(\mu_0\), \(w\) T \(\mu_1\), \(w\) T \(\Sigma_0 w\) and \(w\) T \(\Sigma_1 w\) are all real numbers.

[0016] Consider minimizing \(w\) T \(\Sigma_0 w + w\) T \(\Sigma_1 w\) and maximizing Then the target to be maximized can be obtained as follows:

[0017]

[0018] Define the "within-class scatter matrix":

[0019]

[0020] And the "between-class scatter matrix":

[0021] \(S\) b \(= (\mu_0 - \mu_1)(\mu_0 - \mu_1)\) T (5)

[0022] Then equation (3) can be rewritten as:

[0023]

[0024] Let \(w\) T \(S\) w \(w = 1\), then equation (6) is equivalent to:

[0025]

[0026] Then it can be solved by Lagrange multipliers:

[0027]

[0028] Considering the stability of the numerical solution, in practice, it is usually subjected to singular value decomposition to obtain After that, \(w\) can be calculated according to equation (8).

[0029] Step 3: Construct a metric function to measure the similarity between the source domain and the target domain;

[0030] Given a pair of source domain and target domain, after using the linear discriminant analysis method, an objective function is established to minimize the within-class distance and maximize the between-class distance. The calculation formula is as follows:

[0031] J2 = D w -αD b (9)

[0032] Where D W is the within-class distance, and d b is the between-class distance. The present invention adopts the k-nearest neighbor calculation method and combines a metric function to calculate the within-class and between-class distances of the source domain and the target domain. The data after projection transformation is denoted as n is the number of times a subject is tested, represents the i-th sample, and y i refers to the label of the corresponding sample. If the sample is in the k-th nearest neighbor of the sample , they are considered similar and belong to the same class; otherwise, they do not belong to the same class. The calculation methods of the within-class distance and the between-class distance are as follows:

[0033]

[0034]

[0035] Where the calculation formula of the metric function is as follows:

[0036]

[0037] When is the within-class k1-nearest neighbor of , P ij is equal to 1, otherwise it is 0; when is the between-class k2-nearest neighbor of when ij is equal to 1, otherwise it is 0;

[0038] After the above assumptions, the final problem is to find a suitable metric function d f to calculate the similarity between the source domain and the target domain, and then select the source domain with higher similarity for transfer learning.

[0039] For the samples and The Mahalanobis distance between them can be written as:

[0040]

[0041] Where the M matrix is also called the "metric matrix", and metric learning is to learn M. Note that to ensure that the distance is non-negative and symmetric, M must be a positive semi-definite matrix, that is, there must exist an orthogonal basis P such that M = PPT 。

[0042] Learning the metric matrix M requires setting an objective function. Assuming that the optimization goal is to improve the performance of the nearest neighbor classifier, M can be directly embedded into the evaluation index of the nearest neighbor classifier, and the corresponding M can be obtained by optimizing this performance index. The nearest neighbor classifier mentioned above uses the majority voting method for discrimination, that is, each sample in the neighborhood votes 1, and each sample outside the neighborhood votes 0. Let's replace it with the probability voting method. For any sample The probability that it affects the classification result is:

[0043]

[0044] When i = j, p ij is the largest, and Ω i represents the subscript set of samples belonging to the same category. Obviously, The influence on decreases as the distance between them increases. Taking the maximization of the leave-one-out accuracy as the goal, the leave-one-out accuracy of the overall can be calculated:

[0045]

[0046] Substituting Equation (14) into the above formula, and then considering M = PP T , the optimization goal of the nearest neighbor classifier can be obtained:

[0047]

[0048] Solving Equation (16) can obtain the distance metric matrix M. According to Equation (13), the metric function can be obtained, and then the metric function is used to measure the similarity between each source domain and the target domain to select the most suitable source domain for migration.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] The metric learning method is adopted to select a suitable source domain for migration.

[0051] In order to avoid the negative transfer problem caused by individual source domain data with poor distribution, the present invention studies a source domain selection method based on metric learning, uses a metric learning function to calculate the similarity between the target domain and the source domain, selects a source domain with a higher similarity for transfer learning, and can greatly shorten the time of transfer learning while ensuring high classification accuracy. The purpose of metric learning is to find a suitable metric method to reveal the relationship between data. Therefore, the present invention selects the Mahalanobis distance To measure the distance (similarity) between samples. The metric matrix M is embedded into the evaluation index of k-nearest neighbor classification for learning. After learning the metric function, the distance between the source domain and the target domain is calculated to select the most suitable source domain for transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0053] Figure 1 It is a flowchart of the source domain selection method based on metric learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0055] As Figure 1 shown, the implementation of the present invention mainly includes three steps: (1) performing Euclidean alignment on the original k-th source domain EEG data and the target domain EEG data; (2) using linear discriminant analysis to reduce the dimension of the aligned data, reduce the within-class difference of the samples, and increase the between-class difference of the samples; (3) designing a metric function Calculate the within-class distance D w and the between-class distance D b To measure the similarity between the k-th source domain and the target domain.

[0056] The following will explain each step in detail.

[0057] Step (1): Calibrate and align the original EEG data in the Euclidean space.

[0058] Since the data distributions between different subjects vary greatly, and the large distribution differences will reduce the generalization performance of the classifier. Therefore, in order to make the data distributions between different subjects more similar, the present invention adopts the Euclidean alignment method. Compared with the traditional Riemannian alignment, it has a faster calculation speed and does not require the label information of new subjects.

[0059] Its operation process is as follows: Assume that the data (subjects) of each source domain is D = {x i , y i}, y i ∈ {0, 1}, i = 1, 2, …, n, where n is the number of times a subject is tested, x i represents the i-th sample, y i refers to the label of the corresponding sample. Then, without the need for source domain and target domain labels, the following reference matrix can be calculated:

[0060] Equation (1)

[0061]

[0062] where is the arithmetic mean of all covariance matrices from a single subject. Therefore, the data after Euclidean alignment is denoted as:

[0063] Equation (2):

[0064]

[0065] After alignment, all the average covariance matrices of the subjects are:

[0066] Equation (3):

[0067]

[0068] As can be seen from the above equation, the average covariance matrices of all subjects are equal to the identity matrix after alignment. Therefore, the covariance distributions from different subjects are more similar. In transfer learning, this similar distribution is highly desirable.

[0069] Step 2: Perform linear discriminant analysis using the data after Euclidean alignment.

[0070] Since the data distributions after Euclidean alignment are too similar, if a metric function is directly used to measure the distance (similarity) between the source domain and the target domain, the relative distance between the domains cannot be accurately calculated. Therefore, the present invention adds a linear discriminator between the two to construct a projection space, such that the points of the same-class samples in the projection space are as close as possible, and the points of the different-class samples in the projection space are as far as possible.

[0071] The specific implementation is as follows: Assume that the data after Euclidean alignment is n is the number of times a subject is tested, represents the i-th sample, y i refers to the label of the corresponding sample. Let μ i , Σ i respectively represent the set, mean, and covariance matrix of the i-th class of examples. Denote the projection space as w, then the projections of the centers of the two classes of samples in the projection space are w T μ0 and wT μ1; If all the sample points are projected onto the projection space, the covariance matrices of the two types of samples are w T Σ0w and w T Σ1w. Where w T μ0, w T μ1, w T Σ0w and w T Σ1w are all real numbers.

[0072] If we want the points of the same-class samples in the projection space to be as close as possible, we can make the covariance of the projected points of the same-class samples as small as possible, that is, minimize w T Σ0w + w T Σ1w; and if we want the points of different-class samples in the projection space to be as far as possible, we can make the distance between the class centers as large as possible, that is, maximize Considering both, we can obtain the objective to be maximized:

[0073] Equation (4):

[0074]

[0075] Define the "within-class scatter matrix":

[0076] Equation (5):

[0077]

[0078] And the "between-class scatter matrix":

[0079] Equation (6):

[0080] S b =(μ0 - μ1)(μ0 - μ1) T

[0081] Then Equation (4) can be rewritten as:

[0082] Equation (7):

[0083]

[0084] This is the objective that linear discriminant analysis wants to maximize, that is, the "generalized Rayleigh quotient" of S w and S b . Note that both the numerator and denominator of the above formula are quadratic terms about w, which means that the solution of Equation (7) is independent of the length of w and only related to its direction. Therefore, without loss of generality, let w T S w w = 1, then Equation (7) is equivalent to:

[0085] Equation (8):

[0086]

[0087] Again, by Lagrange multipliers, the above equation is equivalent to:

[0088] Equation (9):

[0089] S b w = λS w w

[0090] where λ is the Lagrange multiplier. Note that the direction of S b w is always μ0 - μ1. Without loss of generality, let

[0091] Equation (10):

[0092] S b w = λ(μ0 - μ1)

[0093] Substituting the above equation into Equation (9), the numerical solution of w can be obtained:

[0094] Equation (11):

[0095]

[0096] Considering the stability of the numerical solution, in practice, singular value decomposition is usually performed on it, that is, let S w = UΣV T , where Σ is a real diagonal matrix, and the elements on its diagonal are the singular values of S w Singular values. Then, from we get where U and V are unitary matrices corresponding to the dimensions of Σ respectively. After obtaining , w can be calculated according to Equation (11). The projection point on w satisfies the goal of minimizing the within-class scatter and maximizing the between-class scatter.

[0097] Step (3): Construct a metric function to measure the similarity between each source domain and the target domain.

[0098] The present invention designs a metric function to calculate the within-class and between-class distances of samples, and then measures the similarity between each source domain and the target domain. The specific process is as Figure 1 shown.

[0099] A good distance should reflect the distance information in the original EEG samples. Given a pair of source domain and target domain, after using the linear discriminant analysis method, an objective function is established to minimize the within-class distance and maximize the between-class distance. The calculation formula is as follows:

[0100] Equation (12):

[0101] J2 = D w - αD b

[0102] Among them, D w is the within-class distance, and D b is the between-class distance. The present invention adopts the k-nearest neighbor calculation method to calculate the within-class and between-class distances of the source domain and the target domain under the metric function. Denote the data after projection transformation as D = n is the number of times a same subject is tested, represents the i-th sample, and y i refers to the label corresponding to the sample. If the sample is in the k-th nearest neighbors of the sample , they are considered similar and belong to the same class; otherwise, they do not belong to the same class. The calculation methods of the within-class distance and the between-class distance are as follows:

[0103] Equation (13):

[0104]

[0105] Equation (14):

[0106]

[0107] Among them, the calculation formula of the metric function is as follows:

[0108] Equation (15):

[0109]

[0110] When is the k1-th within-class nearest neighbor of , P ij is equal to 1, otherwise it is 0; when is the k2-th between-class nearest neighbor of , Q ij is equal to 1, otherwise it is 0. After the above assumptions, the final problem is to find a suitable metric function d f to calculate the similarity between the source domain and the target domain, and then select several source domains with higher similarity for metric learning of the source domain.

[0111] Next, mainly analyze the selection of the metric function d f and its derivation formula. For two r-dimensional samples and The squared Euclidean distance between them can be written as:

[0112] Equation (16):

[0113]

[0114] Among them represents and Distance in the r-th dimension. If it is assumed that different samples have different importance, an attribute weight h can be introduced to obtain the following weighted metric function:

[0115] Equation (17):

[0116]

[0117] where h i ≥0, H = diag(h) is a diagonal matrix, (H) ii = h i . Although H in the above equation can be determined through learning, there can be a better solution. Since the non-diagonal elements of H are 0, this indicates that the coordinate axes are orthogonal, that is, the sample attributes are pairwise independent. However, in the field of motor imagery-based brain-computer interfaces, this assumption often does not hold. For example, the mu rhythm in the 8 - 12 Hz range in the central sulcus area of the cerebral hemisphere and the beta rhythm in the frontal lobe area of the brain not only have a negative correlation but also a positive correlation. This means that the coordinate axes corresponding to the sample attributes are no longer orthogonal. Therefore, H in Equation (17) is replaced by a general positive semi-definite matrix M, and thus the Mahalanobis distance is obtained:

[0118] Equation (18):

[0119]

[0120] where the M matrix is also called the "metric matrix", and metric learning is to learn M. Note that to ensure that the distance is non-negative and symmetric, M must be a positive semi-definite matrix, that is, there must be an orthogonal basis P such that M = PP T .

[0121] Learning the metric matrix M requires setting an objective function. Assuming that the optimization goal is to improve the performance of the nearest neighbor classifier, M can be directly embedded into the evaluation index of the nearest neighbor classifier, and the corresponding M can be obtained by optimizing this performance index.

[0122] The nearest neighbor classifier mentioned above uses the majority voting method during discrimination, that is, each sample in the neighborhood votes 1 and each sample outside the neighborhood votes 0. Let's replace it with the probability voting method. For any sample it has a probability of affecting the classification result as:

[0123] Equation (19):

[0124]

[0125] When i = j, p ij is the largest. Obviously, for The influence decreases as the distance between them increases. With the goal of maximizing the leave-one-out accuracy, the leave-one-out accuracy of can be calculated, that is, the probability that it is correctly classified by all samples except itself:

[0126] Equation (20):

[0127]

[0128] Ω i denotes the set of subscripts of samples belonging to the same class. So the leave-one-out accuracy on the entire sample set is:

[0129] Equation (21):

[0130]

[0131] Substitute Equation (19) into the above formula, and then considering M = PP T , the optimization objective of the nearest neighbor classifier can finally be obtained:

[0132] Equation (22):

[0133]

[0134] Solving Equation (22) can obtain the distance metric matrix M. Further, according to Equation (18), the metric function can be obtained. Then use the metric function to measure the similarity between the source domain and the target domain, so as to select the most suitable source domain for migration.

[0135] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principles and spirits of the present invention, various changes, modifications, substitutions and variations to these embodiments still fall within the protection scope of the present invention.

Claims

1. A source domain selection method based on metric learning, characterized in that: Including the following steps: Step 1: Perform Euclidean alignment on the original EEG data; Assume that the subject data of each source domain is D = {x i , y i}, y i ∈ {0, 1}, i = 1, 2, …, n, where n is the number of times the same subject is tested. Then, without the need for source domain and target domain labels, the following reference matrix can be calculated: wherein is the arithmetic mean of all covariance matrices from a subject, and the data after Euclidean alignment is denoted as: Step 2: Use linear discriminant analysis method to reduce the dimension of the aligned data; Step 3: Construct a metric function to measure the similarity between the source domain and the target domain; Given a pair of source domain and target domain, use the linear discriminant analysis method to establish an objective function to minimize the within-class distance and maximize the between-class distance. The calculation formula is as follows: J2 = D w -αD b (9) Among them, D w is the within-class distance, and D b is the between-class distance. The calculation methods for the within-class distance and the between-class distance are as follows: Where the calculation formula of the metric function is as follows: When is the intra-class k1 nearest neighbor of, P ij equals 1, otherwise 0; When is the inter-class k2 nearest neighbor of, Q ij equals 1, otherwise 0; For samples and The Mahalanobis distance between them can be written as: To ensure that the distance is non - negative and symmetric, M must be a positive semi - definite matrix, i.e., there must exist an orthogonal basis P such that M = PP T , Embed M directly into the evaluation index of the nearest neighbor classifier, and obtain the corresponding M by optimizing the performance index. For any sample It has an impact on The probability of the classification result being affected is: When i = j, p ij is the largest, and Ω i represents the set of subscripts of samples belonging to the same category. Aiming at maximizing the leave-one-out accuracy rate, the leave-one-out accuracy rate of the whole can be calculated as follows: Substitute Equation (14) into the above equation, and then considering \(M = PP\) T , the optimization objective of the nearest neighbor classifier can be obtained: Solving equation (16) can obtain the distance metric matrix M. According to equation (13), the metric function can be obtained. Then, the metric function is used to measure the similarity between each source domain and the target domain to select the most suitable source domain for migration.

2. The source domain selection method based on metric learning according to claim 1, wherein: The specific steps of step two include: The data after Euclidean alignment is i = 1, 2, …, n, where n is the number of times a same subject is tested Let μ i ,Σ i represent the set, mean, and covariance matrix of the i-th class of examples, respectively. Denote the projection space as w, and the projections of the centers of the two types of samples in the space are μ0 and μ1; T μ0 and w T μ1; If projection transformation is performed on all samples, the covariance matrices of the two classes are \(w\) T \(\Sigma_0^w\) and \(w\) T \(\Sigma_1^w\). Considering minimizing \(w\) T \(\Sigma_0^w + w\) T \(\Sigma_1^w\) and maximizing Then the objective to be maximized can be obtained: Define the "within-class scatter matrix": And the "between-class scatter matrix": S b =(μ0 - μ1)(μ0 - μ1) T (5) Then Equation (3) can be rewritten as: Let w T S w If w = 1, then Equation (6) is equivalent to: Furthermore, it can be solved by Lagrange multiplication: Perform singular value decomposition on it to obtain Then, w can be calculated according to Equation (8).

Images