A robust kernel learning method based on maximum mixed cross-correlation entropy criterion

Through the maximum hybrid cross-correlation entropy criterion, the kernel limit learning method is optimized, and the kernel limit learning method is insufficient under noise interference is solved, and the effective regression and classification tasks in complex noise environments are realized, which shows stronger robustness sex.

CN112446506BActive Publication Date: 2024-11-05XIAN SENYUE ELECTRONIC TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910828106.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-09-03
Publication Date
2024-11-05
Estimated Expiration
2039-09-03

AI Technical Summary

Technical Problem

When the nuclear limit learning method contains impulse noise or outlier interference in the data, the learning ability is greatly reduced, making it difficult to effectively complete regression and classification tasks, especially when noise interference is complex in practical applications.

Method used

The maximum hybrid cross-correlation entropy criterion is used as the optimization criterion. The training input data is mapped to the high-dimensional feature space through the Gaussian kernel function, the optimization model is established and the model parameters are solved, and the mapping relationship between the input data and the output data is constructed using a fixed point iteration strategy. , enhance robustness.

Benefits of technology

In the presence of complex noise or abnormal interference, the regression and classification tasks can be effectively completed, which significantly improves performance in noise environments, and shows extremely strong robustness when dealing with non-Gaussian noise.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112446506B_ABST
    Figure CN112446506B_ABST
Patent Text Reader

Abstract

The present application relates to the field of data processing technology, and in particular to a robust kernel learning method based on the maximum mixed mutual correlation entropy criterion. The method comprises: obtaining training input data and training output data; mapping the training input data to a high-dimensional feature space to obtain first training input data; establishing an optimization model according to the first training input data, and solving the model parameters; and establishing a mapping relationship between input data and output data according to the model parameters. The method provided in the present application utilizes the characteristic that the maximum mixed mutual correlation entropy criterion is insensitive to abnormal interference, so that it can still successfully complete regression and classification tasks in the presence of complex noise or abnormal interference.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a robust kernel learning method based on a maximum mixed mutual correlation entropy criterion. Background Art

[0002] Machine learning is a multi-disciplinary subject involving statistics, probability theory, convex analysis, approximation theory, algorithm complexity theory, etc. It mainly studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0003] After years of development, machine learning has produced many branches. One of the most representative methods is the Kernel Based Extreme Learning Machine (KELM). According to the idea of ​​the kernel method, the connection weights and biases between the input layer and the hidden layer of the network are determined by kernel mapping. This not only retains the characteristic of ELM that it does not need to learn the connection weights and biases between the input layer and the hidden layer, but also cleverly avoids the uncertainty caused by random projection. Therefore, KELM can often obtain more stable and generalized performance.

[0004] However, KELM's learning ability is greatly reduced when the data contains impulse noise or outlier interference. This is because KELM uses the Minimum Mean Square Error (MMSE) criterion as the optimization criterion to solve the network output connection weights by default. As we all know, MMSE requires the data to obey the Gaussian assumption. However, practical application systems are often accompanied by various complex noise interferences, making this assumption difficult to guarantee. Therefore, a new method is urgently needed to solve the problem that it is difficult to effectively complete regression and classification tasks in the presence of complex noise or abnormal interference. Summary of the invention

[0005] The present application provides a robust kernel learning method based on the maximum mixed cross-correlation entropy criterion to solve the problem that it is difficult to effectively complete regression and classification tasks in the presence of complex noise or abnormal interference.

[0006] A robust kernel learning method based on maximum mixed cross-correlation entropy criterion, the method comprising:

[0007] Obtain training input data and training output data;

[0008] Mapping the training input data to a high-dimensional feature space to obtain first training input data;

[0009] Establishing an optimization model according to the first training input data, and solving the model parameters;

[0010] A mapping relationship between input data and output data is established according to the model parameters.

[0011] Optionally, the first training input data is a kernel function constructed by a Gaussian kernel function.

[0012] Optionally, the method includes: using the maximum mutual correlation entropy criterion as an optimization criterion to optimize the model.

[0013] Optionally, mapping the training input data to a high-dimensional feature space to obtain first training input data includes:

[0014] according to We get κ(x n ,·), where κ(·,·) represents a kernel function and σ represents the kernel parameter;

[0015] according to get in Represents x n Feature vector after mapping to kernel space;

[0016] according to The first training input data is obtained.

[0017] Optionally, establishing an optimization model according to the first training input data and solving the model parameters includes:

[0018] Establish an optimization model based on the first training data Where β is the model parameter, 0≤λ≤1 is the mixing coefficient, γ≥0 is the regularization factor, σ1 and σ2 are the kernel parameters, and σ1<σ2 is required;

[0019] The fixed point iteration strategy α(t)=f1(α(t-1)) is used to obtain the value of α, where Δ=diag[θ1,…,θ N ],f1(α)=(ΦΦ T +γΔ -1 ) -1 Y;

[0020] According to β=Φ T α gets β.

[0021] Optionally, establishing a mapping relationship between input data and output data according to the model parameters includes:

[0022] according to get

[0023] according to The mapping relationship between the output data is obtained as follows:

[0024] The technical solution provided by this application includes the following beneficial technical effects:

[0025] The present application provides a robust kernel learning method based on the maximum mixed mutual correlation entropy criterion. The method comprises: obtaining training input data and training output data; mapping the training input data to a high-dimensional feature space to obtain first training input data; establishing an optimization model according to the first training input data, and solving the model parameters; and establishing a mapping relationship between input data and output data according to the model parameters. The method utilizes the characteristic that the maximum mixed mutual correlation entropy criterion is insensitive to abnormal interference, so that it can still successfully complete regression and classification tasks in the presence of complex noise or abnormal interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1 A flowchart of a robust kernel learning method based on the maximum mixed cross-correlation entropy criterion is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] Mixture Correntropy (MC) is a local proximity measure in the kernel space based on multi-Gaussian kernel functions. Given two arbitrary random variables X and Y, the MC between them can be expressed as

[0029]

[0030] Where E represents the expected operation, represents S Gaussian kernels with different kernel parameters, represents S combination parameters (0≤λ s ≤1 and ). In practical applications, usually only a limited number of samples are available, so the cross-correlation entropy can be estimated by the following formula:

[0031]

[0032] For the convenience of derivation, only the case of S = 2 is considered here, so the above formula can be simplified to

[0033]

[0034] In practical applications, we can solve the model parameters by maximizing MC. For example, to estimate the model parameters of a linear system, we can model the problem as

[0035] in and Y represent the actual and ideal system outputs respectively, and ω represents the system parameter to be solved. This is a typical optimization problem under the Maximum Mixture Correntropy Criterion (MMCC). Since the MMCC criterion is extremely robust to non-Gaussian noise interference (especially impulse noise), it is very popular in robust machine learning and signal processing.

[0036] The present application provides a robust kernel learning method based on the maximum mixed cross-correlation entropy criterion, the method comprising:

[0037] S1: Obtain training input data and training output data.

[0038] A common problem in machine learning is to find the training input data X = [x1 T , x2 T , ..., x N T ] T With the training output data Y = [y1 T ,y2 T ,...,y N T ] T , find a function relationship f that approximately satisfies y n =f(x n ), n=1,2,…,N. Therefore, the method provided in this application is also to find a functional relationship between the training output data and the training input data. First, it is necessary to obtain the trained input data and output data. In this application, the training input data and the training output data are both given.

[0039] S2: Mapping the training input data to a high-dimensional feature space to obtain first training input data.

[0040] The specific steps are as follows:

[0041] According to the Gaussian kernel function Construct a kernel function κ(·,·), where exp represents an exponential function with the natural constant e as the base; x and x' represent two random variables; and σ represents the kernel parameter.

[0042] The training input data X = [x1 T ,x2 T,…,x N T ] T Mapping to a high-dimensional feature space to get the original training input data in Indicates that x n The feature vector after being mapped to the kernel space. In this application, the Gaussian kernel function is selected to construct κ(·,·) because the Gaussian kernel function is easy to operate.

[0043] S3: Establish an optimization model according to the first training input data, and solve the model parameters.

[0044] The specific steps are as follows:

[0045] Using the maximum cross-correlation entropy criterion as the optimization criterion, we get

[0046] in β represents the unknown model parameters.

[0047] Taking the derivative of J(β) with respect to β, we get

[0048] in

[0049] For ease of representation, the above formula is written in the following matrix form: where Δ=diag[θ1,…,θ N ] is a weighted diagonal matrix.

[0050] Because the optimal β should satisfy So β=(Φ T ΔΦ+γI) -1 Φ T ΔY

[0051] The above formula β = Φ T (ΦΦ T +γΔ -1 ) -1 Y deformation is β = Φ T α

[0052] The transformed β is equivalent to the linear combination of the input data after mapping, that is, β = Φ T α, where α=(ΦΦ T +γΔ -1 ) -1 Y;

[0053] And because ΦΦ T The value of can be obtained by kernel technique To obtain, therefore

[0054]

[0055] The above process transforms the problem of solving β into solving α, and β = Φ T Substituting α into Δ, each diagonal element in Δ can be re-expressed as

[0056]

[0057] In the above formula The value of can also be obtained by using the kernel technique Obtain

[0058]

[0059] Let α=f1(α), where f1(α)=(ΦΦ T +γΔ -1 ) -1 Y. Using the classic fixed-point iteration strategy, we have

[0060] α(t+1)=f1(α(t))

[0061] Where α(t) represents the solution obtained after t iterations. In practical applications, the initial value of α can be set to 0, and then the formula α(t+1)=f1(α(t)) can be repeatedly used until the algorithm converges.

[0062] The first training input data

[0063] S4: Establish a mapping relationship between input data and output data according to model parameters.

[0064] The specific steps are as follows:

[0065] according to get

[0066] according to The mapping relationship between the output data is obtained as follows:

[0067] In order to verify the performance of KELM-MMCC, we first consider a simple nonlinear function fitting task, namely

[0068] f(x)=0.8exp(-0.2x)sin(10x)

[0069] Using the formula f(x) = 0.8exp(-0.2x)sin(10x), 500 sets of training samples and 500 sets of test samples are generated. In order to make the data more "real", different types of noise interference will be added to the output of the training samples, while ensuring that the test samples are clean. The noise model adopted here is v i =(1-ηi )A i +η i B i , where η i is a Bernoulli random process with a success rate of 0.03, A i It follows a Gaussian distribution with a mean of 0 and a variance of 0.0001. i , we consider two cases: (1) uniform distribution on [-2,2]; (2) binary distribution with equal probability of taking values ​​2 or -2. For ease of comparison, the testing root mean square error (TRMSE) is defined as

[0070]

[0071] Where N te Indicates the number of test samples.

[0072] Table 1 shows the comparison results of the testing root mean square error (TRMSE) of KELM and KELM-MMCC and the related parameter settings. In order to ensure a fair comparison and reduce random errors, all parameters are determined by the "five-fold cross validation method", and all experimental results are averaged based on 20 independent runs. From Table 1, it can be seen that the proposed KELM-MMCC can maintain excellent performance (very small TRMSE value) in two different noise interference environments, while the performance of KELM is far worse than that of KELM-MMCC, which shows that the proposed new method has a stronger ability to cope with complex noise interference.

[0073] Table 1 TRMSE results and related parameter settings of KELM and KELM-MMCC

[0074]

[0075]

[0076] Furthermore, KELM-MMCC was used for electroencephalogram (EEG) classification tasks. The experimental data used the IIa public dataset of the 4th International Brain-Computer Interface Competition. It contains EEG data measured from 9 healthy subjects. Each subject had to perform 576 motor imagery tasks, including left hand imagined movement (category one), right hand imagined movement (category two), tongue imagined task (category three), and foot movement imagined task (category four). While he (she) was performing the motor imagery task, an EEG cap with 22 electrodes was used to collect his (her) EEG signals. The collected EEG signals were divided into two groups, one for training and the other for testing. Specifically for each subject "A1"-"A9", their corresponding training trial numbers and test trial numbers were both 288. In order to remove data segments unrelated to motor imagery, for each training or test trial data, we selectively intercepted the 0.5s-2.5s data segment after each instruction was issued as a valid sample. These samples are filtered through a 5th order Butterworth bandpass filter with a start and end frequency of 8Hz and 30Hz. On this basis, Common Spatial Space (CSP) combined with One-Versus-One (OVO) technology is used to extract effective features of these samples. Finally, 24 spatial domain filters are used to construct feature vectors. These feature vectors and corresponding labels will be used to train the classifier and test the classification performance of the classifier.

[0077] In addition to KELM, Support Vector Machine (SVM) and Extreme Learning Machine (ELM) will also be used to perform classification tasks on the public dataset of the 3rd International Brain-Computer Interface Competition IVa. Among these algorithms, SVM directly calls the toolbox provided by MATLAB, and ELM is downloaded directly from the homepage of Professor Huang Guangbin of Nanyang Technological University, Singapore. In order to obtain the best classification accuracy of the corresponding algorithm, we also use the "five-fold cross validation method" to determine their optimal parameter configuration.

[0078] Table 2 shows the best test classification accuracy of different algorithms on the public dataset IIa of the 4th International Brain-Computer Interface Competition. The classification accuracy corresponding to each subject in the table is averaged based on 20 Monte Carlo runs. It can be seen that KELM-MMCC can achieve high classification accuracy on most subjects, and the average classification accuracy achieved on all subjects is also the highest.

[0079] Table 2 The best test classification accuracy of different algorithms on the public dataset IIa of the 4th International Brain-Computer Interface Competition

[0080]

[0081]

[0082] The present application provides a robust kernel learning method based on the maximum mixed mutual correlation entropy criterion. The method comprises: obtaining training input data and training output data; mapping the training input data to a high-dimensional feature space to obtain first training input data; establishing an optimization model according to the first training input data, and solving the model parameters; and establishing a mapping relationship between input data and output data according to the model parameters. The method utilizes the characteristic that the maximum mixed mutual correlation entropy criterion is insensitive to abnormal interference, so that it can still successfully complete regression and classification tasks in the presence of complex noise or abnormal interference.

[0083] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0084] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features disclosed herein.

[0085] It should be understood that the present application is not limited to what has been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A robust kernel learning method based on the maximum mixed cross-correlation entropy criterion, characterized in that: The method is used for electroencephalogram classification and specifically comprises the following steps: Acquire training input data and training output data; wherein the types of the training input data and the training output data include left hand imaginary movement, right hand imaginary movement, tongue imaginary task and feet imaginary task; the training input data and the training output data are EEG signals collected when performing the left hand imaginary movement, right hand imaginary movement, tongue imaginary task and feet imaginary task; Mapping the training input data to a high-dimensional feature space to obtain first training input data; the first training input data is a kernel function constructed by a Gaussian kernel function; Mapping the training input data to a high-dimensional feature space to obtain first training input data includes: according to We get κ(x n ,·), where κ(·,·) represents a kernel function, σ represents the kernel parameter; exp represents the exponential function with the natural constant e as the base, and x and x' represent two random variables of σ; according to get in Represents x n Feature vector after mapping to kernel space; according to Obtaining the first training input data; Establishing an optimization model according to the first training input data and solving the model parameters; optimizing the model using the maximum cross-correlation entropy criterion as the optimization criterion; Establishing an optimization model according to the first training input data and solving the model parameters includes: Establish an optimization model based on the first training data Among them, β is the model parameter, 0≤λ≤1 is the mixing coefficient, γ≥0 is the regularization factor, σ1 and σ2 are the kernel parameters, and σ1<σ2 is required. y n =f(x n ),n=1,2,…,N: The fixed point iteration strategy α(t)=f1(α(t-1)) is used to obtain the value of α, where Δ=diag[θ1,…,θ N ], a=(ΦΦ T +gD -1 ) -1 Yes, α(t) represents the solution obtained at the tth iteration, f1(α)=(ΦΦ T +γΔ -1 ) -1 Y; According to β=Φ T α gets β; A mapping relationship between input data and output data is established according to the model parameters.

2. The robust kernel learning method based on the maximum mixed cross-correlation entropy criterion according to claim 1, characterized in that: Establishing a mapping relationship between input data and output data according to the model parameters includes: according to get according to The mapping relationship between the output data is obtained as follows: