Rapid projection-based biplane support vector regression method

Abstract

The invention discloses a novel and rapid projection-based biplane support vector regression method. Specifically, based on a biplane support vector regression (TSVR) thought, two non-parallel hyperplanes are constructed to optimize regression boundaries, correlation of intra-class data is considered, the intra-class data distance is made to be minimum through projection, least square equality constraints are introduced, a convex quadratic programming problem is simplified into a solving problem of two small-scale linear equation sets, and the problem is solved through the two small-scale linear equation sets. Therefore, the algorithm complexity is reduced. Due to simplification of the optimization problem, the training speed of the model provided by the invention is higher than that of the traditional SVR, and the method is suitable for processing a large-scale data set and has good practical application potential.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, and more particularly, to a fast projection-based two-plane support vector regression method. BACKGROUND

[0002] Support Vector Machine (SVM) is a widely used supervised learning algorithm in the field of pattern recognition and machine learning. SVM is a binary classification model, whose core idea is to find an optimal hyperplane in the feature space that can separate different classes of data points as much as possible and has the maximum margin. This optimal hyperplane is also called decision boundary. SVM maximizes the margin to improve the generalization ability of the model. Support vectors are the key to the SVM model, and SVM only cares about the support vectors closest to the hyperplane, and is insensitive to other data, so it has strong anti-interference ability to noise data.

[0003] The existing Two-Plane Support Vector Regression Machine (TSVR) and Support Vector Regression Machine (SVR) are both regression methods based on support vector machines, but they have some significant differences in algorithm principles, optimization objectives and solving processes. The core idea of SVR is to find an optimal hyperplane that minimizes the distance error of all training sample points to the hyperplane and maximizes the margin. TSVR is a regression method developed on the basis of SVR, which proposes two non-parallel hyperplanes to approximate the data. TSVR solves two smaller quadratic programming problems instead of a larger quadratic programming problem like SVR, thereby reducing the computational complexity. TSVR and SVR both have good generalization ability, but the specific performance may vary depending on the data set and parameter settings. In some cases, TSVR may exhibit better generalization ability because it approximates the data with two non-parallel hyperplanes, which may be more flexible in capturing the distribution characteristics of the data.

[0004] However, the above-mentioned traditional Support Vector Regression Machine (SVR) minimizes the prediction error by finding an optimal hyperplane, which performs well in complex scenarios such as small sample and high-dimensional data, but still faces challenges when dealing with large-scale and nonlinear data. SUMMARY

[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a new fast projection-based two-plane support vector regression method. This method is based on the two-plane support vector regression idea, and constructs two non-parallel hyperplanes to optimize the regression boundary. At the same time, in order to exploit the correlation of intra-class data, projection is used to minimize the distance of intra-class data, and finally least squares equality constraints are introduced to optimize the objective function, thereby reducing the algorithm complexity.

[0006] To achieve the above object, the application provides the following technical scheme: a fast projection-based double-plane support vector regression method, characterized by the following steps: Step one, obtaining original data ; Step two, constructing a basic double-plane support vector regression model; Step three, introducing projection to realize intra-class data mapping transformation and mine intra-class relationship; Step four, introducing least square equation constraint to simplify the convex quadratic programming problem into two smaller linear equation set solving problems; Step five, optimizing the double-plane support vector regression model based on step four; Step six, designing a decision classification hyperplane.

[0007] As a further improvement of the application, the step two of constructing a basic double-plane support vector regression model has the following objective function of the model: (1) (2) In addition, it also includes lower bound function and upper bound function: and upper bound function , wherein, m represents an adjustment parameter, is a relaxation vector.

[0008] As a further improvement of the application, the step three of introducing projection to realize intra-class data mapping transformation and mine intra-class relationship can obtain the following objective function: (3) and (4) Wherein, is an adjustment parameter for balancing the weights of each term, finds the projection input point with the minimum variance.

[0009] As a further improvement of the application, the step four of introducing least square equation constraint to simplify the convex quadratic programming problem into two smaller linear equation set solving problems can obtain the following objective function: (5) and (6).

[0010] As a further improvement of the application, the constraint condition in the objective function is brought in to obtain an unconstrained optimization problem, which is as follows: ​ .

[0011] As a further improvement of the present application, in the objective function formula (5) and formula (6), partial derivatives of and are taken to be 0, and let: = =0, , we get: (7); Partial derivatives of and are taken to be 0, and let: = =0, , we get: (8).

[0012] As a further improvement of the present application, the optimization of the double plane support vector regression model in the fifth step is specifically: Based on formula (7) in the fourth step, we get and are (9) Similarly, based on formula (8) in the fourth step, we get and are (10).

[0013] As a further improvement of the present application, the decision function in the sixth step is as follows: . (11).

[0014] The present application has the following advantages: 1) The present application optimizes the support vector regression model, simplifies the solving process, and converts the complex convex quadratic programming problem of the double plane support vector regression model based on projection into a simple linear equation set solving problem by minimizing the sum of squares of errors, thereby improving the solving efficiency of the model.

[0015] 2) The present application has good application potential in the field of large-scale data processing, has the advantage of fast real-time processing, and the semantic analysis of data class labels can ensure that the regression model has good fitting performance. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The flow chart of the fast projection-based double plane support vector regression method of the present application. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the embodiments shown in the accompanying drawings.

[0018] Reference Figure 1 As shown in this embodiment, a novel and fast projection-based biplane support vector regression method is presented. This method is based on the biplane support vector regression concept, constructing two non-parallel hyperplanes to optimize the regression boundary. Simultaneously, to uncover the correlation between intra-class data, projection is used to minimize the distance between intra-class data. Finally, least squares equality constraints are introduced to optimize the objective function, reducing the algorithm's complexity. The specific implementation steps are as follows: Step 1: First, obtain the data used for regression analysis in the actual application scenario, such as, but not limited to, the following scenarios: historical data on stock prices, exchange rates, etc. in the field of market trend prediction; various financial indicators and market data in the field of financial risk assessment; image restoration data, used to restore damaged or blurred images and improve image quality; audio and video signals in signal processing, used for audio signal enhancement and filtering, etc.; gene expression data, to identify nonlinear relationships between genes and predict changes in gene expression levels; protein structure data, by analyzing the relationship between protein sequences and known structures; historical data on air quality, water quality, etc. in environmental monitoring; historical traffic data in traffic flow prediction, etc. Specifically, describe the data values ​​and predicted values ​​in the above scenarios as datasets. .

[0019] Step Two: Construct the basic Two-Plane Support Vector Regression (TSVR) model. This model is the foundational model for regression analysis, based on the principle of structural risk minimization. It requires solving two relatively small quadratic programming problems to obtain two regression hyperplanes with smaller fitting errors. The specific objective function is as follows: (1) (2) This also includes upper and lower bound functions: lower bound function and upper bound function , middle Indicates the adjustment parameter. It is the relaxation vector. From models (1) and (2), it can be seen that TSVR aims to find the upper bound function of the lower bound function. The lower bound of the band and the upper bound function The goal is to include as many sample output values ​​as possible within the band, which causes TSVR to lose its sparsity and increase bandwidth. Different choices can also affect the fitting accuracy.

[0020] Step 3: Introduce projection to achieve intra-class data mapping transformation, explore intra-class relationships, and design the objective function as follows: (3) and (4) where, is a tuning parameter that balances the weights of each term, serves to find the projection input point that minimizes the variance. The second term in the objective function is minimized is equivalent to finding the normal vector that maximizes the empirical correlation between the projection input and the target. The third term in the objective function is optimized by , the sum of the estimated values of the training points. Specifically, the objective function maximizes . Thus, optimizing the third term in the objective function makes the function as large as possible while making the function as small as possible. The constraint term in model (3) requires that the estimated value of the training point obtained be less than the response value of the training point. That is, the response value of the training point should be greater than the estimated value obtained . Otherwise, a slack variable is introduced to measure the error. The constraint term in model (4) requires that the estimated value of the training point obtained be less than the response value of the training point. That is, the response value of the training point should be greater than the estimated value obtained . Otherwise, a slack variable is introduced to measure the error.

[0021] Step four: The constraint conditions of the regression problem in equation (3) and equation (4) are inequality constraints, and the input data needs to satisfy certain boundary conditions. In order to realize fast and efficient modeling, further introduce least square equality constraints, convert inequality constraints into equality constraints, that is, require the sum of the squares of the distances of all sample points to the regression hyperplane to be equal to a constant. This transformation makes the solving process more concise and efficient, so that two smaller linear equation systems are obtained to solve the problem; (5) and (6) where, the fourth term in the objective function is a quadratic penalty. The final model uses two non-parallel hyperplanes with two equality constraints to project the data, optimize the performance of the regression model, and improve the accuracy and efficiency of data fitting.

[0022] Further: the constraint condition in the above objective function (5) is substituted into the objective function to obtain the following unconstrained optimization problem, wherein, For the unconstrained optimization problem of formula (5), the necessary condition for finding the optimal solution is that the gradient is zero, that is, for and The partial derivative is 0, and let: = =0, It can be obtained that: (7) For the unconstrained optimization problem of formula (6), the necessary condition for finding the optimal solution is that the gradient is zero, that is, for and The partial derivative is 0, and let: = =0, It can be obtained that: (8) Step five: based on formula (7) in step four, so that and is (9) Similarly, it can be obtained that and is (10) Formula (9) and formula (10) depict the final projection-based double-plane support vector regression hyperplane.

[0023] Step six: the obtained decision function is: . (11) The final prediction value is the average value of the two hyperplanes, and the above learning strategy has the advantages of rapidness and efficiency.

[0024] The above only describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application shall be considered as falling within the protection scope of the present application. It should be noted that for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application shall also be considered as falling within the protection scope of the present application.

Claims

1. A fast projection-based biplane support vector regression method characterized in that: The method comprises the following steps: Step one, obtaining raw data ; Step two, constructing a basic double-plane support vector regression model; Step three, introducing a projection to realize an intra-class data mapping transformation and mine intra-class relationships; Step four, introducing a least square equation constraint to simplify a convex quadratic programming problem into two linear equation group solving problems of smaller scale; Step five, realizing optimization of the double-plane support vector regression model based on step four; Step six, designing a decision classification hyperplane.

2. The fast projection-based biplane support vector regression method of claim 1, wherein: In the step two, the objective function of the basic double-plane support vector regression model is specifically as follows: (1)； (2)； wherein, also included are upper and lower bound functions: lower bound function and upper bound function , wherein denotes the adjustment parameter, is the relaxation vector.

3. The fast projection-based biplane support vector regression method of claim 2, wherein: In the step three, the objective function of the intra-class data mapping transformation and the mining of intra-class relationships is as follows: (3)； and (4)； wherein, is a tuning parameter that balances the weights, The role of is to find the projection input point that has the smallest variance.

4. The fast projection-based biplane support vector regression method of claim 3, wherein: In the step four, the objective function of the least square equation constraint is as follows: (5)； and (6)。 5. The fast projection-based biplane support vector regression method of claim 4, wherein: The constraint condition in the objective function is brought in to obtain an unconstrained optimization problem, which is specifically as follows: 。 6. The fast projection-based biplane support vector regression method of claim 5, wherein: In the objective function (5) and (6), the partial derivatives of and are set to 0, and we have: = =0, and (7)； For and Take partial derivative to 0, let: = =0, In the step five, the optimization of the double-plane support vector regression model is realized in the following specific manner: (8)。 7. The fast projection-based biplane support vector regression method according to claim 6, characterized in that: In the step six, the decision function is as follows: Based on formula (7) in step four, thus obtaining and is (9)； By analogy, based on formula (8) in step four, we obtain and is (10)。 8. The fast projection-based biplane support vector regression method of claim 7, wherein: ​ . (11)。

Images