A style transfer method based on similarity matrix to preserve the local structure of target domain
By introducing similarity matrix and Geodesic Flow Kernel into the style transfer method, combined with K-mean clustering, the maintenance of the target domain local structure is optimized, and the problems of insufficient migration performance and long calculation time in the existing technology are solved, and more efficient EEG signal emotion recognition classification.
Patent Information
- Application Number
- CN202111564016.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-12-20
AI Technical Summary
The existing style transfer methods are difficult to effectively maintain the local structure of the target domain in the field of emotion recognition, resulting in insufficient migration performance, especially on the EEG signal dataset, which is long calculation time and low classification accuracy.
Using a style transfer method based on similar matrix, features are extracted through Geodesic Flow Kernel and K mean clustering, mapping matrices A and b are calculated, target domain local structure is maintained, dependence on a large amount of unknown label data is reduced, migration process is optimized, and hyperparameters is used. Controls migration degree and structural weight.
It significantly reduces the calculation time, improves the classification accuracy, is suitable for EEG signal data sets, enhances migration performance, reduces the demand for unknown label data, and optimizes the operating efficiency of the original STM framework.
Smart Images

Figure BDA0003421542620000021 
Figure BDA0003421542620000022 
Figure BDA0003421542620000033
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of transfer learning and relates to the application of a style transfer method based on maintaining the local structure of a target domain in the field of emotion recognition. Background Art
[0002] Transfer learning aims to achieve twice the result with half the effort by transferring knowledge. Transfer learning leverages similarities between data, tasks, or models to apply models and knowledge learned in previous domains to new domains, enabling the new domain to make decisions based on minimal knowledge. Its superior performance has attracted numerous scholars and researchers both domestically and internationally to conduct extensive research on transfer learning. Currently, leading methods include Transfer Component Analysis (TCA), Joint Distribution Adaptation (JDA), Balanced Distribution Adaptation (BDA), Manifold Embedded Distribution Alignment (MEDA), Manifold Embedding Knowledge Transfer (MEKT), and Style Transfer Mapping (STM). Classical classification or fitting methods, such as Support Vector Machines (SVM), K-Nearest Neighbor (KNN), and Neural Networks, are then applied to the target domain data.
[0003] Style transfer methods directly map an origin distribution to a destination distribution. Currently, style transfer methods are widely used and have been studied in fields such as image recognition and emotion recognition. Many improvements based on style transfer methods have been made, most notably the improvement proposed by Li et al. in emotion recognition based on domain prototype cluster centers and class averages. Inspired by the improved STM method by Li et al., this paper proposes a novel STM method using a similarity matrix to preserve the local structure of the target domain. Experimental results on emotion datasets demonstrate the effectiveness of this method, which significantly improves the transfer performance of the original STM method. Summary of the Invention
[0004] In response to the shortcomings of existing style transfer methods, the present invention proposes a style transfer method based on similarity matrix that preserves the local structure of the target domain.
[0005] A similarity matrix-based style transfer method that preserves the local structure of the target domain, the method specifically comprising the following steps:
[0006] Step 1: The features extracted from different frequency bands of the EEG signals obtained by each subject through the brain-computer interface are used as each domain; the domain that has not been used as the target domain is selected as the target domain T, and the remaining domains are used as the source domain S;
[0007] Step 2: Take a certain number of samples from T that contain different classes as samples with known labels T0, and the rest as target domain samples with unknown labels T1;
[0008] Step 3: Use the Geodesic Flow Kernel method to migrate S and T0 in turn and predict the category of T0. Output the classification accuracy and use it as the final weight of the corresponding source domain;
[0009] Step 4: Perform K-means clustering on each type of S in turn to find N center points of each type;
[0010] Step 5: Calculate the bi-norm value of each sample and each center point of T0, and select the center point with the smallest value as the mapping destination of the sample.
[0011] Step 6: Based on the mapping destination point obtained in Step 5, the mapping matrix A0, b0 is calculated using the improved style transfer method through equations (3) to (8);
[0012] The objective function of the improved style transfer method is
[0013]
[0014]
[0015] Among them, ‖·‖2 refers to the second norm of the quantity, is the Frobenius norm of the matrix; R m×m The dimension is m×m; n is the number of samples; A, b are mapping matrices; the degree of migration is controlled by hyperparameters β and γ, and the Controls the weight of preserving the local structure of the target domain based on the similarity matrix; f is the confidence; o is the mapping origin; d is the mapping target point; I is the identity matrix; σ is a scaling parameter; where Tr(·) is the trace of the calculation matrix, is a self-set parameter; formula (1) has a closed-form solution
[0016] A=QP -1 ,
[0017] in
[0018]
[0019]
[0020]
[0021]
[0022]
[0023] Step 7: According to the corresponding mapping matrix A0, b0, T0 is mapped to T 00
[0024] Step 8: Calculate T 00 The two-norm value of each sample and each center point is calculated, and the center point with the smallest value is selected as the updated mapping destination of the sample.
[0025] Step 9: Still using T0, and based on the mapping destination point obtained in Step 8, use the improved style transfer method to calculate the mapping matrix A1, b1.
[0026] Step 10: Map T1 according to the corresponding mapping matrix A1, b1, expressed as T 10 ; and use the classifier trained in the corresponding S to classify T 10 Classify and output the domain pair T 10 Category prediction. S's predictions about T1 are weighted and summarized based on the weights. The final predicted category is output based on the sum of the weights, and the classification accuracy is obtained.
[0027] Step 11: Determine whether each domain has become the target domain. If not, return to Step 1. If yes, continue to the next step.
[0028] Step 12: Calculate the average classification accuracy of each domain after it is used as the target domain, and use it as the classification accuracy of the new style transfer method on the dataset.
[0029] As a preference, the hyperparameters are set to
[0030]
[0031] As a preference, the features extracted from different frequency bands of the EEG signals obtained by each subject through the brain-computer interface are used as each domain; the domain that has not been used as the target domain is selected as the target domain T, and the remaining domains are used as the source domain S;
[0032] Specifically:
[0033] Since it is relatively easy to obtain EEG signals from different people, it meets the multi-source requirements under the optimization framework, and the dimensions of the EEG signals after feature extraction are appropriate, so this method is more suitable for EEG signals. The EEG signal sample set used for the experiment contains 3-category emotional EEG data of 15 subjects. Each subject obtained 3394 310-dimensional samples containing 3 categories of different emotions. The data obtained from the 15 subjects were used as 15 different domains Y1-Y 15 , select the domain that has not been used as the target domain as the target domain T, and the remaining domains as the source domain S (S1-S 14 ).
[0034] Preferably, f is the confidence level, which can participate in parameter updating under the STM framework of Li's experiment.
[0035] Preferably, in the following framework of the present invention, f is set to a constant of 1 and σ is set to 1.
[0036] The STM used by Li et al. uses a large amount of unknown label target domain data to update the parameters in the STM, so the calculation time is lengthy and cannot adapt well to actual needs. Since the objective function proposed by the present invention adds a regularization term that maintains the local structure of the target domain, it can reduce the demand for a large amount of unknown label target domain data to a certain extent. Based on this improvement, the present invention further limits the mapping matrices A, b, so that the classification framework of the original STM can be further optimized at the same time, the program running time is reduced, and the step of using a large amount of unknown label target domain data in the original STM to update the parameters in the STM is deleted. A small amount of known label target domain data is directly used for the improved migration calculation, and experiments have proved that this method can not only greatly reduce the calculation time, but also improve the final classification accuracy. Compared with the STM framework used in the experiment by Li et al., this optimization lacks the confidence parameter f i The update of f i It has always been 1, which reduces the accuracy to a certain extent. However, due to the addition of the regularization term to maintain the local structure of the target domain, the overall accuracy is improved, and ultimately the result of time optimization and accuracy improvement is achieved. DETAILED DESCRIPTION
[0037] A similarity matrix-based style transfer method that preserves the local structure of the target domain, the method specifically comprising the following steps:
[0038] Step 1: The features extracted from different frequency bands of the EEG signals obtained by each subject through the brain-computer interface are used as each domain;
[0039] The sample set used for the experiment includes 3-category emotional EEG data of 15 subjects. Each subject obtained 3394 310-dimensional samples containing 3 categories of different emotions. The data obtained from the 15 subjects are used as 15 different domains Y1-Y 15 , select the domain that has not been used as the target domain as the target domain T, and the remaining domains as the source domain S (S1-S 14 ).
[0040] Step 2: Take a certain number of samples from T that contain different classes as samples with known labels T0, and the rest as target domain samples with unknown labels T1;
[0041] Step 3: Use the Geodesic Flow Kernel method to migrate S and T0 in turn and predict the category of T0. Output the classification accuracy and use it as the final weight of the corresponding source domain;
[0042] Step 4: Perform K-means clustering on each type of S in turn to find N center points of each type;
[0043] Step 5: Calculate the bi-norm value of each sample and each center point of T0, and select the center point with the smallest value as the mapping destination of the sample.
[0044] Step 6: Based on the mapping destination point obtained in Step 5, the mapping matrix A0, b0 is calculated using the improved style transfer method through equations (5) to (11);
[0045] The objective function of the improved style transfer method is
[0046]
[0047]
[0048] Among them, ‖·‖2 refers to the second norm of the quantity, is the Frobenius norm of the matrix; R m×m The dimension is m×m; n is the number of samples; A, b are mapping matrices; the degree of migration is controlled by the hyperparameters β and γ, while the weight of maintaining the local structure of the target domain based on the similarity matrix is controlled by the hyperparameter ζ; f is the confidence; o is the mapping origin; d is the mapping target point; I is the identity matrix; σ is a scaling parameter; where Tr(·) is the trace of the calculation matrix, is a self-set parameter; Formula (3) has a closed-form solution
[0049] A=QP -1 ,
[0050] in
[0051]
[0052]
[0053]
[0054]
[0055]
[0056]
[0057] Step 7: According to the corresponding mapping matrix A0, b0, T0 is mapped to T 00
[0058] Step 8: Calculate T 00 The two-norm value of each sample and each center point is calculated, and the center point with the smallest value is selected as the mapping destination of the sample.
[0059] Step 9: Based on the mapping destination point obtained in Step 8, the mapping matrix A1, b1 is calculated using the improved style transfer method.
[0060] Step 10: Map T1 according to the corresponding mapping matrix A1, b1, expressed as T 10 ; and use the classifier trained in the corresponding S to classify T 10 Classify and output the domain pair T 10 Category prediction. S's predictions about T1 are weighted and summarized based on the weights. The final predicted category is output based on the sum of the weights, and the classification accuracy is obtained.
[0061] Step 11: Determine whether each domain has become the target domain. If not, return to Step 1. If yes, continue to the next step.
[0062] Step 12: Calculate the average classification accuracy of each domain after it is used as the target domain, and use it as the classification accuracy of the new style transfer method on the dataset.
[0063] The following is a comparison of the advantages and disadvantages of the new style transfer method proposed in this invention and the previous typical transfer methods for transfer classification on the Shanghai Jiao Tong University SEED dataset, as shown in Table 1:
[0064] Traditional migration methods STM method New STM method Calculation time short long short Accuracy high high high
[0065] Table 1
[0066] The present invention will be further analyzed below with reference to specific examples.
[0067] The DE_LDS feature sample data in the SEED dataset was used to test the accuracy of the migration method. The final results are shown in Table 2.
[0068]
[0069] Table 2
Claims
1. A style transfer method based on similarity matrix to preserve the local structure of the target domain, characterized in that: The method specifically comprises the following steps: Step 1: The features extracted from different frequency bands of the EEG signals obtained by each subject through the brain-computer interface are used as each domain; the domain that has not been used as the target domain is selected as the target domain T, and the remaining domains are used as the source domain S; Step 2: Take a certain number of samples of different categories from T as samples with known labels T0, and the rest as target domain samples with unknown labels T1; Step 3: Use the Geodesic Flow Kernel method to migrate S and T0 in turn and predict the category of T0, output the classification accuracy and use it as the final weight of the corresponding source domain; Step 4: Perform K-means clustering on each type of S in turn to find N center points of each type; Step 5: Calculate the bi-norm value of each sample and each center point of T0, and select the center point with the smallest value as the mapping destination of the sample; Step 6: Based on the mapping destination point obtained in Step 5, the mapping matrix A0, b0 is calculated using the improved style transfer method through equations (3) to (8); The objective function of the improved style transfer method is where ‖·‖2 refers to the second norm of the quantity, is the Frobenius norm of the matrix; R m×m The dimension is m×m; n is the number of samples; A, b are mapping matrices; the degree of migration is controlled by hyperparameters β and γ, and the weight of maintaining the local structure of the target domain based on the similarity matrix is controlled by hyperparameter ζ; f is the confidence; o is the mapping origin; d is the mapping target point; I is the identity matrix; σ is a scaling parameter; where Tr(·) is the trace of the calculation matrix, is a self-set parameter; Formula (1) has a closed-form solution in Step 7: According to the corresponding mapping matrix A0, b0, T0 is mapped to T 00 ; Step 8: Calculate T 00 The bi-norm value of each sample and each center point, and the center point with the smallest value is selected as the mapping destination point after the sample is updated; Step 9: Still use T0, and according to the mapping destination point obtained in Step 8, use the improved style transfer method to calculate the mapping matrix A1, b1; Step 10: Map T1 according to the corresponding mapping matrix A1, b1, expressed as T 10 ; and use the classifier trained in the corresponding S to classify T 10 Classify and output the source domain S to T 10 Category prediction; weighted summary of S's predictions about T1 based on weights; output the final prediction category based on the sum of weights and obtain the classification accuracy; Step 11: Determine whether each domain has been the target domain. If not, return to Step 1. If yes, proceed to the next step. Step 12: Calculate the average classification accuracy of each domain as the target domain, and use it as the classification accuracy of the new style transfer method.
2. According to claim 1, a style transfer method based on similarity matrix to maintain the local structure of the target domain: the features extracted from different frequency bands of the EEG signals obtained by each subject through the brain-computer interface are used as each domain; the domain that has not been used as the target domain is selected as the target domain T, and the remaining domains are selected as the source domain S; Specifically: The sample set used for the experiment contains 3-category emotional EEG data of 15 subjects; each subject obtained 3394 310-dimensional samples containing 3 categories of different emotions; the data obtained from the 15 subjects were used as 15 different domains Y1 to Y 15 , select the domain that has not been used as the target domain as the target domain T, and the remaining domains as the source domain S, S1 to S 14 .
3. According to the style transfer method based on similarity matrix and maintaining the local structure of the target domain as described in claim 1: f is the confidence, which participates in parameter updating under the STM framework of Li's experiment.
4. According to the style transfer method based on similarity matrix and maintaining the local structure of the target domain as described in claim 1: f is set to a constant of 1 and σ is set to 1.
Citation Information
Patent Citations
Electroencephalogram signal classification method based on multi-source manifold embedding migration
CN112684891A
Electroencephalogram emotion migration method based on deep subdomain association adaptation network
CN113157096A
Image classification method based on reliable weighted optimal transport (RWOT)
US20210390355A1
