A method for constructing a non-parallel structure support vector machine

Abstract

The present invention discloses a construction method of a non-parallel structure support vector machine. On the basis of a traditional parallel support vector machine (parallel SVM), by adding the least squares term of the sample, an additional empirical risk minimization term and an added bias constraint term are added to the original optimization problem, and two non-parallel hyperplanes are obtained respectively to form a new additional empirical risk minimization non-parallel support vector machine. The following steps are included: data preprocessing, solving the Lagrange multiplier of the positive class hyperplane, solving the Lagrange multiplier of the negative class hyperplane, solving the parameters of the positive and negative classification hyperplanes, and determining the category of new data points. The present invention adopts the above-mentioned construction method of a non-parallel structure support vector machine, proposes a new non-parallel vector machine algorithm, and further improves the classification accuracy of hyperspectral images on the basis of the algorithm itself to obtain better classification performance.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a construction method of a non-parallel structure support vector machine. Background Art

[0002] Support Vector Machine (SVM) was proposed by Vapnik et al. and is applicable to pattern recognition and other fields. The characteristic of SVM is that it can take into account both empirical risk and structural risk, that is, supervised learning is achieved by finding a hyperplane that can both ensure classification accuracy and maximize the interval between two types of data. SVM has some good characteristics, such as kernel skills, sparsity, and global solution. Due to its solid theoretical foundation and good generalization, it is widely used in remote sensing image classification. The support vector machine classification model assumes that the positive and negative category boundaries are parallel. However, for actual remote sensing data, this assumption is difficult to hold, which affects the generalization ability of the model. To address this problem, Jayadeva et al. proposed Twin Support Vector Machine (TWSVM). The goal of TWSVM is to find a pair of non-parallel (parallel can be regarded as a special state of non-parallel) hyperplanes. The data points of each category are close to one of the two non-parallel hyperplanes and far away from the other. The category to which the sample belongs is determined by comparing the distance from the sample to the two hyperplanes. TWSVM is particularly successful, but it still has obvious shortcomings: the TWSVM model only considers empirical risk but not structural risk, which affects its generalization performance, so that in many cases its classification effect is not as good as that of traditional support vector machines. Therefore, TWSVM is not very effective when used directly for hyperspectral image classification. Some scholars have combined it with other algorithms to use it on hyperspectral data. This paper aims to propose a new non-parallel vector machine algorithm to further improve the classification accuracy of hyperspectral images based on the algorithm itself.

[0003] In view of the above situation, this paper constructs a non-parallel support vector machine model by adding the least squares term of samples and the empirical risk minimization additional term on the basis of the traditional parallel support vector machine, namely, Additional Empirical Risk Minimization Nonparallel Support Vector Machine, AERM-NSVM, hereinafter referred to as this patented method. Summary of the invention

[0004] The purpose of the present invention is to provide a construction method of a non-parallel structure support vector machine, propose a new non-parallel vector machine algorithm, and further improve the classification accuracy of hyperspectral images on the basis of the algorithm itself to achieve better classification performance.

[0005] To achieve the above object, the present invention provides a method for constructing a non-parallel structure support vector machine, comprising the following steps:

[0006] S1, data preprocessing;

[0007] S2, solve the Lagrange multiplier of the positive hyperplane;

[0008] S3. Solve the Lagrange multiplier of the negative hyperplane;

[0009] S4, solving the parameters of the positive and negative classification hyperplanes;

[0010] S5. Determine the category of new data points.

[0011] Preferably, the data preprocessing in step S1 specifically includes the following steps:

[0012] (1) Read m binary classification n-dimensional training data sets and perform standardization. Obtain an m×n training data sample matrix C, and read the label information as a vector y;

[0013] (2) Separate the training samples according to the positive and negative label information to obtain m + ×n matrices A and m - ×n matrix B;

[0014] (3) Convert the label item y into a diagonal matrix Y.

[0015] Preferably, solving the Lagrange multiplier of the positive class hyperplane in step S2 specifically includes the following steps:

[0016] (1) Construct m + Dimensional unit vector I + , we can get the matrix P from formula (1): + ;

[0017]

[0018] (2) From formula (2), we get the matrix Q + ;

[0019] Q + =-e T (2);

[0020] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (3) + ;

[0021] J + =[C3×e T ] (3);

[0022] (4) Construct m +Dimension all 1 vector e + , we can get the matrix K from formula (4): + ;

[0023]

[0024] (5) From formula (5), we can get the Lagrange multiplier vector α = (α1, ..., α m )and

[0025]

[0026] Or from formula (6), we can get the Lagrange multiplier vector α=(α1,...,α m )and

[0027]

[0028] Preferably, solving the Lagrange multiplier of the positive class hyperplane in step S2 specifically includes the following steps:

[0029] (1) Construct m + Dimensional unit vector I + , we can get the matrix P from formula (7): + ;

[0030]

[0031] (2) From formula (8), we get the matrix Q + ;

[0032]

[0033] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (9): + ;

[0034]

[0035] (4) Construct m + Dimension all 1 vector e + , we can get the matrix K from formula (10): + ;

[0036]

[0037] (5) From formula (11), we can get the Lagrange multiplier vector α = (α1, ..., α m )and

[0038]

[0039] Or from formula (12), we can get the Lagrange multiplier vector α=(α1,...,α m )and

[0040]

[0041] Preferably, solving the Lagrange multiplier of the negative hyperplane in step S3 specifically includes the following steps:

[0042] (1) Construct an m-dimensional identity matrix I - , we can get the matrix P from formula (13): - ;

[0043]

[0044] (2) From formula (14), we get the matrix Q - ;

[0045] Q _ =-e T (14);

[0046] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (15) _ ;

[0047] J _ =[C4×e T ] (15);

[0048] (4) Construct an m-dimensional all-one vector e - , we can get the matrix K from formula (16): _ ;

[0049]

[0050] (5) From formula (17), we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0051]

[0052] Or from formula (18) we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0053]

[0054] Preferably, solving the Lagrange multiplier of the negative hyperplane in step S3 specifically includes the following steps:

[0055] (1) Construct m - Dimensional identity matrix I - , we can get the matrix P from formula (19): - ;

[0056]

[0057] (2) From formula (20), we get the matrix Q - ;

[0058]

[0059] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (21) _ ;

[0060] J _ =[C4×e T ] (twenty one);

[0061] (4) Construct an m-dimensional all-one vector e - , we can get the matrix K from formula (22): _ ;

[0062]

[0063] (5) From formula (23), we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0064]

[0065] Or from formula (24) we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0066]

[0067] Preferably, solving the parameters of the positive and negative classification hyperplanes in step S4 specifically includes the following steps:

[0068] (1) The normal vector ω of the positive classification hyperplane is obtained from formula (25): + ;

[0069] ω + =-A T λ+C T Y T α (25);

[0070] (2) The positive classification hyperplane offset b is obtained from formula (26): + ;

[0071]

[0072] (3) From formula (27), we get the negative classification hyperplane normal vector ω - ;

[0073] ω - =B T θ+C T Y T γ (27);

[0074] (4) The negative classification hyperplane offset b is obtained from formula (28): - ;

[0075]

[0076] Preferably, solving the parameters of the positive and negative classification hyperplanes in step S4 specifically includes the following steps:

[0077] (1) The normal vector ω of the positive classification hyperplane is obtained from formula (29): + ;

[0078] ω + =-A T λ+C T Y T α (29);

[0079] (2) The positive classification hyperplane offset b is obtained from formula (30): + ;

[0080]

[0081] (3) From formula (31), we get the negative classification hyperplane normal vector ω - ;

[0082] ω - =B T θ+C T Y T γ (31);

[0083] (4) The negative classification hyperplane offset b is obtained from formula (32): - ;

[0084]

[0085] Preferably, the new data point category determination in step S5 specifically includes the following steps:

[0086] (1) Obtain test data x, and use formula (33) to obtain the Euclidean distance from x to the positive classification hyperplane;

[0087]

[0088] (2) The Euclidean distance from x to the negative classification hyperplane is obtained by formula (34);

[0089]

[0090] (3) Determine d + and d _ Which is smaller if d + <d - Then the x category is positive, otherwise it is negative.

[0091] Preferably, the construction method is performed using a linear approach, and if used in a nonlinear situation, the expansion method is consistent with the linear approach;

[0092] Description of the binary classification case. If used in a multi-classification case, the expansion method is the same as that of the binary classification case.

[0093] Therefore, the present invention adopts the above-mentioned non-parallel structure support vector machine construction method and proposes a new non-parallel vector machine algorithm to further improve the classification accuracy of hyperspectral images on the basis of the algorithm itself, so as to achieve better classification performance.

[0094] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 The classification hyperplane of the method of the present invention, wherein the origin of part A represents the positive sample point, the origin of part B represents the negative sample point, the line with triangles represents the two parallel planes obtained by solving the positive sample optimization problem, and the line with circles represents the two parallel planes obtained by solving the negative sample optimization problem;

[0096] Figure 2 The image is the classification result of the Pavia Center hyperspectral image, where a is the ground truth, b is SVM, c is TWSVM, and d is the method of the present invention;

[0097] Figure 3 This is the classification result image of the Pavia University hyperspectral image, where a is the ground truth, b is SVM, c is TWSVM, and d is the method of the present invention. DETAILED DESCRIPTION

[0098] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0099] Example 1

[0100] The present invention provides a method for constructing a non-parallel structure support vector machine, comprising the following steps:

[0101] S1. Data preprocessing, specifically including the following steps:

[0102] (1) Read m binary classification n-dimensional training data sets and perform standardization. Obtain an m×n training data sample matrix C, and read the label information as a vector y;

[0103] (2) Separate the training samples according to the positive and negative label information to obtain m + ×n matrices A and m - ×n matrix B;

[0104] (3) Convert the label item y into a diagonal matrix Y.

[0105] S2. Solving the Lagrange multiplier of the positive hyperplane includes the following steps:

[0106] (1) Construct m + Dimensional unit vector I + , we can get the matrix P from formula (1): + ;

[0107]

[0108] (2) From formula (2), we get the matrix Q + ;

[0109] Q + =-e T (2);

[0110] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (3) + ;

[0111]

[0112] (4) Construct m + Dimension all 1 vector e + , we can get the matrix K from formula (4): + ;

[0113]

[0114] (5) From formula (5), we can get the Lagrange multiplier vector α = (α1, ..., α m )and

[0115]

[0116] Or from formula (6), we can get the Lagrange multiplier vector α=(α1,...,α m )and

[0117]

[0118] S3, solving the Lagrange multiplier of the negative hyperplane, specifically including the following steps:

[0119] (1) Construct an m-dimensional identity matrix I - , we can get the matrix P from formula (13): - ;

[0120]

[0121] (2) From formula (14), we get the matrix Q - ;

[0122] Q _ =-e T (14);

[0123] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (15) _ ;

[0124] J _ =[C4×e T ] (15);

[0125] (4) Construct an m-dimensional all-one vector e - , we can get the matrix K from formula (16): _ ;

[0126]

[0127] (5) From formula (17), we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0128]

[0129] Or from formula (18) we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0130]

[0131] S4, solving the parameters of the positive and negative classification hyperplane, specifically including the following steps:

[0132] (1) The normal vector ω of the positive classification hyperplane is obtained from formula (25): + ;

[0133] ω + =-A T λ+C T Y T α (25);

[0134] (2) The positive classification hyperplane offset b is obtained from formula (26): + ;

[0135]

[0136] (3) From formula (27), we get the negative classification hyperplane normal vector ω - ;

[0137] ω - =B T θ+C T Y T γ (27);

[0138] (4) The negative classification hyperplane offset b is obtained from formula (28): - ;

[0139]

[0140] S5, new data point category determination, specifically including the following steps:

[0141] (1) Obtain test data x, and use formula (33) to obtain the Euclidean distance from x to the positive classification hyperplane;

[0142]

[0143] (2) The Euclidean distance from x to the negative classification hyperplane is obtained by formula (34);

[0144]

[0145] (3) Determine d + and d _ Which is smaller if d + <d - Then the x category is positive, otherwise it is negative.

[0146] Example 2

[0147] The present invention provides a method for constructing a non-parallel structure support vector machine, comprising the following steps:

[0148] S1. Data preprocessing, specifically including the following steps:

[0149] (1) Read m binary classification n-dimensional training data sets and perform standardization. Obtain an m×n training data sample matrix C, and read the label information as a vector y;

[0150] (2) Separate the training samples according to the positive and negative label information to obtain m + ×n matrices A and m - ×n matrix B;

[0151] (3) Convert the label item y into a diagonal matrix Y.

[0152] S2. Solving the Lagrange multiplier of the positive hyperplane includes the following steps:

[0153] (1) Construct m + Dimensional unit vector I + , we can get the matrix P from formula (7): + ;

[0154]

[0155] (2) From formula (8), we get the matrix Q + ;

[0156]

[0157] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (9): + ;

[0158] J + =[C3×e T ] (9);

[0159] (4) Construct m + Dimension all 1 vector e + , we can get the matrix K from formula (10): + ;

[0160]

[0161] (5) From formula (11), we get the Lagrange multiplier vector α = (α1, ..., α m )and

[0162]

[0163] Or from formula (12), we can get the Lagrange multiplier vector α=(α1,...,α m )and

[0164]

[0165] S3, solving the Lagrange multiplier of the negative hyperplane, specifically including the following steps:

[0166] (1) Construct an m-dimensional identity matrix I - , we can get the matrix P from formula (19): - ;

[0167]

[0168] (2) From formula (20), we get the matrix Q - ;

[0169]

[0170] (3) Construct an m-dimensional all-one vector e, and obtain the matrix J by formula (21) _ ;

[0171] J _ =[C4×e T ] (twenty one);

[0172] (4) Construct m - Dimension all 1 vector e - , we can get the matrix K from formula (22): _ ;

[0173]

[0174] (5) From formula (23), we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0175]

[0176] Or from formula (24) we can get the Lagrange multiplier vector θ = (θ1, ..., θ m ) and γ=(γ1,...,γ m- );

[0177]

[0178] S4, solving the parameters of the positive and negative classification hyperplane, specifically including the following steps:

[0179] (1) The normal vector ω of the positive classification hyperplane is obtained from formula (25): + ;

[0180] ω + =-A T λ+C T Y T α (25);

[0181] (2) The positive classification hyperplane offset b is obtained from formula (26): + ;

[0182]

[0183] (3) From formula (27), we get the negative classification hyperplane normal vector ω - ;

[0184] ω - =B T θ+C T Y T γ (27);

[0185] (4) The negative classification hyperplane offset b is obtained from formula (28): - ;

[0186]

[0187] The construction methods of the present invention are all carried out in a linear manner. If used in a nonlinear situation, the expansion method is consistent with the linear method; the description of the binary classification situation, if used in a multi-classification situation, the expansion method is consistent with the binary classification situation.

[0188] like Figure 1 , each quadratic programming problem can obtain a pair of parallel hyperplanes like SVM, but they all shift toward the corresponding samples. The lines with triangles represent the two parallel planes obtained by solving the positive sample optimization problem, and the lines with circles represent the two parallel planes obtained by solving the negative sample optimization problem. At this time, the straight line close to the positive sample point and the straight line close to the negative sample point are taken as a pair of classification decision hyperplanes of AERM-NSVM.

[0189] In order to illustrate the effectiveness of the present invention, the following experimental demonstration is carried out.

[0190] 1. Pavia Center Dataset

[0191] The Pavia Center dataset is acquired by the ROSIS sensor in Pavia, northern Italy. The number of spectral bands of Pavia Center is 102. Pavia Center is an image of 1096×1096 pixels, which contains 9 categories. The sample division of the training set and test set is shown in Table 1:

[0192] Table 1 Number of samples in each category of Pavia center scene

[0193]

[0194]

[0195] The classification results are as follows Figure 2 The classification accuracy is shown in Table 2.

[0196] Table 2 Classification results of Pavia Center hyperspectral image

[0197]

[0198] The Pavia Center dataset has a large amount of data, and equal amounts of samples were taken for training. As can be seen from Table 2, the classification results of TWSVM in the Pavia Center dataset are still slightly lower than those of SVM, and the Kappa coefficient is also lower than that of SVM. The classification accuracy of the method of the present invention exceeds that of the standard SVM, and the Kappa coefficient is also higher than that of SVM.

[0199] 2. Pavia University Dataset

[0200] The Pavia University dataset is acquired by the ROSIS sensor in Pavia, northern Italy. The number of spectral bands of the University of Pavia is 103. The University of Pavia is 610×610 pixels and contains 9 categories. The sample division of the training set and the test set is shown in Table 3:

[0201] Table 3 Number of samples in each category at Pavia University

[0202]

[0203]

[0204] The classification results are as follows Figure 3 The analysis accuracy is shown in Table 4.

[0205] Table 4 Classification results of Pavia University hyperspectral images

[0206]

[0207] As can be seen from Table 4, the classification accuracy of TWSVM is higher than that of SVM in the Pavia University dataset, and the Kappa coefficient is close to that of SVM. It is more suitable for non-parallel classification planes on this dataset. The classification accuracy of the method of the present invention is more excellent than that of the standard SVM and TWSVM. The accuracy of the method of the present invention is 1.05% higher than that of the standard SVM, and the Kappa coefficient is 1.29% higher than that of the SVM. The accuracy of the method of the present invention is 0.95% higher than that of the standard SVM, and the Kappa coefficient is 1.16% higher than that of the SVM. It shows that the method of the present invention with structural risk minimization can achieve better results than TWSVM which only performs empirical risk minimization.

[0208] Therefore, the present invention adopts the above-mentioned non-parallel structure support vector machine construction method and proposes a new non-parallel vector machine algorithm to further improve the classification accuracy of hyperspectral images on the basis of the algorithm itself, so as to achieve better classification performance.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for constructing a non-parallel structure support vector machine, characterized in that: The following steps are involved: S1. Data preprocessing, the data is hyperspectral image data; specifically includes the following steps: (1) Read binary classification indivual dimensional training data set and perform standardization to obtain The training data sample matrix C is read as the label information vector y; (2) Separate the training samples according to the positive and negative label information to obtain The matrix A and The matrix B; (3) Convert the label item y into a diagonal matrix Y; S2, solve the Lagrange multiplier of the positive hyperplane; The Lagrange multiplier for solving the positive hyperplane in S2 specifically includes: (1) Construction Dimensional unit vector , according to the formula Get the matrix ; ； (2) According to the formula Get the matrix ; ； (3) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (4) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (5) According to the formula , we get the Lagrange multiplier vector and ; ； Or by the formula , we get the Lagrange multiplier vector and ; ； Or the Lagrange multiplier for solving the positive hyperplane in S2 specifically includes: (1) Construction Dimensional unit vector , according to the formula Get the matrix ; ； (2) According to the formula Get the matrix ; ； (3) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (4) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (5) According to the formula , we get the Lagrange multiplier vector and ; ； Or by the formula , we get the Lagrange multiplier vector and ; ； S3. Solve the Lagrange multiplier of the negative hyperplane; The Lagrange multiplier of the negative hyperplane is solved in S3, specifically including: (1) Construction Dimensional identity matrix , according to the formula Get the matrix ; ； (2) According to the formula Get the matrix ; ； (3) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (4) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (5) According to the formula Get the Lagrange multiplier vector and ; ； Or by the formula Get the Lagrange multiplier vector and ; ； Or solving the Lagrange multiplier of the negative hyperplane in S3 specifically includes: (1) Construction Dimensional identity matrix , according to the formula Get the matrix ; ； (2) According to the formula Get the matrix ; ； (3) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (4) Construction Dimensional all 1 vector , according to the formula Get the matrix ; ； (5) According to the formula Get the Lagrange multiplier vector and ; ； Or by the formula Get the Lagrange multiplier vector and ; ； S4, solving the parameters of the positive and negative classification hyperplanes; The parameters of the positive and negative classification hyperplanes are solved in S4, specifically including: (1) According to the formula Get the normal vector of the positive classification hyperplane ; ； (2) According to the formula Get the positive classification hyperplane offset ; ； (3) According to the formula Get the negative classification hyperplane normal vector ; ； (4) According to the formula Get the negative classification hyperplane offset ; ； Or the parameters of the positive and negative classification hyperplanes are solved in S4, specifically including: (1) According to the formula Get the normal vector of the positive classification hyperplane ; ； (2) According to the formula Get the positive classification hyperplane offset ; ； (3) According to the formula Get the negative classification hyperplane normal vector ; ； (4) According to the formula Get the negative classification hyperplane offset ; ； S5. Determine the category of new data points.

2. The method for constructing a non-parallel structure support vector machine according to claim 1, characterized in that: The new data point category determination in step S5 specifically includes the following steps: (1) Obtaining test data , according to the formula get Euclidean distance to the positive classification hyperplane; ； (2) According to the formula get Euclidean distance to the negative classification hyperplane; ； (3) Judgment and Which is smaller if The The category is positive, otherwise it is negative.

Images