Sequence feature selection method based on expanded generalized regression neural network
Through the sequence feature selection method based on the extended generalized regression neural network, the k-fold cross-validation method and target loss optimization technology are used to solve the problem of feature redundancy and high computing cost in feature selection, and the feature selection effect with high accuracy and low computing cost is achieved.
Patent Information
- Application Number
- CN202411833172.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-04-22
AI Technical Summary
Among the existing feature selection methods, the filtering method can easily lead to feature redundancy and low accuracy, while the embedding method requires complex algorithm parameter optimization, resulting in high computational cost.
The sequence feature selection method based on the extended generalized regression neural network is adopted, and the data set is divided by the k-fold cross-validation method. The extended generalized regression neural network is used to calculate the average prediction deviation, generate the target loss, and repeatedly find the feature variable that minimizes the target loss until the target loss is no longer reduced, and an optimal feature subset is generated.
The optimal number of feature variables is achieved adaptively, which enhances the operability and pertinence of the algorithm, improves the accuracy of the algorithm, and does not require parameter optimization, and can be flexibly embedded in the framework of integrated learning methods.
Smart Images
Figure CN119862931A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of feature selection, for example, to a sequence feature selection method based on an extended generalized regression neural network. Background Art
[0002] Feature Selection is also called Feature Subset Selection (FSS) or Attribute Selection. It refers to selecting N features from the existing M features to optimize the specific indicators of the system. It is the process of selecting some of the most effective features from the original features to reduce the dimension of the data set. It is an important means to improve the performance of learning algorithms and a key data preprocessing step in pattern recognition. For a learning algorithm, good learning samples are the key to training the model.
[0003] At present, the commonly used feature selection methods are filtering and embedding. Among them, the basic idea of the filtering method is to calculate the amount of information relative to the prediction result for each feature respectively, obtain the target result, and then sort them from large to small, and output the first few features according to the preset number. The filtering method evaluates each feature independently, which may ignore the interaction between features. However, the correlation between features is often high. In other words, one set of features can be replaced by another set of features, so it is easy to have redundant features and low accuracy. The embedding method first uses certain machine learning algorithms and models for training to obtain the weight coefficients of each feature, and selects features from large to small according to the coefficients. In some cases, such as when using complex models or large-scale data sets, the embedding method requires complex algorithm parameter optimization, resulting in high computational costs.
[0004] Therefore, a sequence feature selection method based on an extended generalized regression neural network is needed to solve the problems that filtering feature selection easily leads to feature redundancy and low accuracy, and embedding feature selection requires complex algorithm parameter optimization.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.
[0007] The present disclosure provides a sequence feature selection method based on an extended generalized regression neural network, including:
[0008] Divide the original data set into a training set and a validation set according to a k-fold cross-validation method, and generate k groups of training sets and a cross-validation set corresponding to each group of training sets; wherein the original data set includes multiple groups of original data, and each group of original data includes B feature variables and corresponding actual responses;
[0009] Initializing a B-dimensional feature subset, wherein the B-dimensional feature subset includes k groups of training sets;
[0010] Based on the extended generalized regression neural network, the average prediction deviation of all cross-validation sets is calculated according to the k training sets to generate the target loss;
[0011] Repeatedly search for the feature variables that minimize the target loss and delete them from the B-dimensional feature subset until the target loss no longer decreases, generating the optimal feature subset.
[0012] In one embodiment, dividing the original data set into a training set and a validation set according to a k-fold cross-validation method, generating k groups of training sets and a cross-validation set corresponding to each group of training sets, includes:
[0013] Divide the original data set into k training samples on average;
[0014] For each training sample, use it as a cross-validation set, and use the remaining k-1 training samples among the k training samples as training sets.
[0015] In one embodiment, the method of calculating the average prediction deviation of all cross-validation sets based on k training sets based on the extended generalized regression neural network to generate the target loss includes:
[0016] For each training set, the extended generalized regression neural network is trained using the training set to generate an extended generalized regression neural network model of the current training set;
[0017] According to the characteristic variables in the cross-validation set corresponding to the current training set, the extended generalized regression neural network model based on the current training set predicts the target response;
[0018] The predicted deviation between the target response and the actual response was calculated based on the preset Pearson correlation coefficient;
[0019] The average of all prediction deviations is calculated to generate the average prediction deviation, and the average prediction deviation is recorded as the target loss.
[0020] In one embodiment, repeatedly searching for feature variables that minimize the target loss and deleting them from the B-dimensional feature subset until the target loss no longer decreases, thereby generating an optimal feature subset, includes:
[0021] Delete certain feature variables from the current feature subset in turn and calculate the target loss; wherein, define the B-dimensional feature subset as the initial current feature subset;
[0022] When the target loss is reduced, find the feature variable corresponding to the minimum target loss; delete the feature variable from the current feature subset to generate the optimal feature subset;
[0023] Update the current feature subset to the preferred feature subset and repeat all the above steps until the target loss no longer decreases, and take the final generated preferred feature subset as the optimal feature subset.
[0024] The sequence feature selection method based on the extended generalized regression neural network provided in the embodiments of the present disclosure can achieve the following technical effects:
[0025] It can adaptively obtain the optimal number of feature variables, enhance the operability and pertinence of the algorithm, and improve the accuracy of the algorithm. At the same time, the algorithm does not require parameter optimization and can be flexibly and conveniently embedded in the framework of integrated learning methods.
[0026] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:
[0028] Figure 1 is a schematic diagram of a sequence feature selection method based on an extended generalized regression neural network provided in an embodiment of the present disclosure;
[0029] Figure 2 It is a schematic diagram of a sequence feature selection method of an extended generalized regression neural network using 3-fold cross validation as an example provided in an embodiment of the present disclosure.
[0030] Figure 3 It is a ladder diagram of soil moisture estimation accuracy using different feature selection methods provided by the embodiment of the present disclosure; wherein a is a ladder diagram of soil moisture estimation accuracy using the RReliefF method; b is a ladder diagram of soil moisture estimation accuracy using the LS method; c is a ladder diagram of soil moisture estimation accuracy using the MDA method; d is a ladder diagram of soil moisture estimation accuracy using the PCC method; Figure e is a ladder diagram of soil moisture estimation accuracy using the MRMR method; Figure f is a ladder diagram of soil moisture estimation accuracy using the EBFS method;
[0031] Figure 4It is a soil moisture map estimated by different feature selection methods provided by the embodiment of the present disclosure; wherein, a is a soil moisture map estimated by the RReliefF method; b is a soil moisture map estimated by the LS method; c is a soil moisture map estimated by the MDA method; d is a soil moisture map estimated by the PCC method; Figure e is a soil moisture map estimated by the MRMR method; Figure f is a soil moisture map estimated by the EBFS method;
[0032] Figure 5 It is a scatter plot of soil moisture and measured values using different feature selection methods provided by the embodiment of the present disclosure; wherein, a is a scatter plot of soil moisture and measured values using the RReliefF method; b is a scatter plot of soil moisture and measured values using the LS method; c is a scatter plot of soil moisture and measured values using the MDA method; d is a scatter plot of soil moisture and measured values using the PCC method; Figure e is a scatter plot of soil moisture and measured values using the MRMR method; Figure f is a scatter plot of soil moisture and measured values using the EBFS method;
[0033] Figure 6 It is the interpretation of the characteristic variables provided in the embodiments of the present disclosure. DETAILED DESCRIPTION
[0034] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.
[0035] The terms "includes," "comprises," and "having" and any variations thereof, are intended to cover a non-exclusive inclusion.
[0036] Unless otherwise stated, the term "plurality" means two or more.
[0037] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.
[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
[0039] Aiming at the shortcomings that the soil moisture sensitive feature selection of filtering method easily leads to feature redundancy and low accuracy, and the feature selection of embedding method requires complex algorithm parameter optimization, a sequence feature selection method based on extended generalized regression neural network (EGRNN) is proposed based on the basic framework of sequence feature selection, namely the backward sequence feature selection method (EBSFS).
[0040] Specifically, refer to Figure 1 and Figure 2 , a sequence feature selection method based on an extended generalized regression neural network provided in an embodiment of the present disclosure includes:
[0041] S101, dividing the original data set into a training set and a validation set according to a k-fold cross-validation method, generating k groups of training sets and a cross-validation set corresponding to each group of training sets; wherein the original data set includes multiple groups of original data, and each group of original data includes B feature variables and corresponding actual responses.
[0042] S102, initializing a K-dimensional feature subset, where the K-dimensional feature subset includes k groups of training sets.
[0043] Reference Figure 2 , in the K-dimensional feature subset here, K == B. That is, this step can actually be written as, initializing the B-dimensional feature subset.
[0044] S103, based on the extended generalized regression neural network, calculate the average prediction deviation of all cross-validation sets according to the k training sets to generate the target loss.
[0045] S104, repeatedly searching for feature variables that minimize the target loss and deleting them from the B-dimensional feature subset until the target loss no longer decreases, thereby generating an optimal feature subset.
[0046] The target loss is generated by using the prediction model based on EGRNN and the k-fold cross validation method, and the feature variable that minimizes the target loss is found in the iteration, and the variable is deleted from the B-dimensional feature subset, so as to continuously update the current feature subset until the optimal feature subset is found. Among them, the feature variable in the optimal feature subset is the feature variable finally selected.
[0047] In this way, EBSFS can adaptively obtain the optimal number of feature variables according to the evaluation criteria, which enhances the operability and pertinence of the algorithm and improves the accuracy of the algorithm. At the same time, the algorithm does not require parameter optimization and can be flexibly and conveniently embedded in the framework of ensemble learning methods.
[0048] In one embodiment, the original data set is divided into a training set and a validation set according to a k-fold cross-validation method to generate k groups of training sets and a cross-validation set corresponding to each group of training sets, including: dividing the original data set into k training samples on average; taking each training sample as a cross-validation set, and taking the remaining k-1 training samples of the k training samples as training sets.
[0049] Using k-fold cross validation to divide the training set and validation set helps prevent overfitting in the process of training data.
[0050] In one embodiment, an average prediction deviation of all cross-validation sets is calculated based on k groups of training sets based on an extended generalized regression neural network to generate a target loss, including: for each group of training sets, using the training set to train an extended generalized regression neural network to generate an extended generalized regression neural network model of the current training set; predicting a target response based on the extended generalized regression neural network model of the current training set according to feature variables in the cross-validation set corresponding to the current training set; calculating the prediction deviation between the target response and the actual response according to a preset Pearson correlation coefficient; calculating the average of all prediction deviations to generate an average prediction deviation, and recording the average prediction deviation as the target loss.
[0051] In this embodiment, the relevant evaluation criteria and objective function are defined by means of the Pearson correlation coefficient, thereby generating the target loss. The evaluation criteria are defined as follows:
[0052] First, we build the EGRNN model. It is known that (X, Y) is n sets of training samples, and the sample feature variables are X = {x1, x2, ..., x n}( is B-dimensional feature), the target is Y={y1,y2,…,y n Assume is unknown data, for the input variable of unknown data Based on the original generalized regression neural network (GRNN), the Manhattan distance d is calculated i , the formula is as follows:
[0053]
[0054] In order to model the local data structure and reduce the computational complexity, the k-nearest neighbor algorithm (k-NN) is introduced into GRNN to infer the sparse Gaussian kernel P i :
[0055]
[0056] Among them, σ is the smoothness parameter that controls the fitness level of GRNN. Usually the smoothness parameter is optimized by cross-validation, which is time-consuming and laborious. To this end, this paper designs an adaptive adjustment method, as shown in the following formula:
[0057]
[0058] Among them, σ opt is the optimal value of σ; For training samples X and unknown data X u The min(·) function returns the minimum value of each column of the input matrix.
[0059] In order to model the local data structure and reduce the computational complexity, this paper introduces the k nearest neighbor algorithm into GRNN to generate k nearest neighbor sets:
[0060] Ω={min(D)} i＝1，2，...，k
[0061] According to σ opt The sparse Gaussian kernel is calculated from the k nearest neighbor sets:
[0062]
[0063] This formula reduces the dimension of the Gaussian kernel matrix from m×n to m×k, and the computational complexity is reduced to O(nklog(n+k) / 2), thereby constructing the EGRNN model as follows:
[0064]
[0065] in, is the weighted sum of neurons in the pattern layer; is the sum of neurons in the pattern layer.
[0066] Then, after the EGRNN model is built, the prediction deviation ξ of all cross-validation sets is calculated based on the given feature subset. i (i=1,2,3):
[0067]
[0068] Among them, the given feature subset here is a given cross-validation set, It is the result predicted by the EGRNN model according to the given cross-validation set; the function corr represents the calculation of the Pearson correlation coefficient.
[0069] The objective function is defined as calculating the average prediction deviation F of all cross-validation sets based on a given feature subset. The formula is as follows:
[0070]
[0071] In one embodiment, the feature variables that minimize the target loss are repeatedly searched and deleted from the B-dimensional feature subset until the target loss is no longer reduced, generating an optimal feature subset, including:
[0072] Delete certain feature variables from the current feature subset in turn and calculate the target loss; wherein, define the B-dimensional feature subset as the initial current feature subset;
[0073] When the target loss is reduced, find the feature variable corresponding to the minimum target loss; delete the feature variable from the current feature subset to generate the optimal feature subset;
[0074] Update the current feature subset to the preferred feature subset and repeat all the above steps until the target loss no longer decreases, and take the final generated preferred feature subset as the optimal feature subset.
[0075] When repeatedly searching for the feature variable that minimizes the target loss, an iterative loop is entered. Specifically, when entering the jth iterative loop, from the current feature subset Γ j Delete a certain feature variable in turn and calculate the target loss F j (F j =F), find the characteristic variable b* that minimizes the target loss:
[0076]
[0077] Delete b* from the current feature subset and update the optimal feature subset Γ j :
[0078] Γ j =Γ j / b*
[0079] Among them, the symbol "Γ j / b * " represents the j Delete the element b* from .
[0080] Repeatedly search for the characteristic variable b* that minimizes the objective loss function. If a characteristic variable Γ is deleted j If it does not decrease any more, the operation is terminated; otherwise, the next cycle is continued. After the operation is terminated, the optimal feature subset Γ is output. opt =Γ j .
[0081] In order to better verify and illustrate the technical effects adopted in the present invention, this embodiment selects multiple different feature selection methods to evaluate the optimal number of features for soil moisture estimation, compares the quantitative difference in the optimal number of features evaluated by multiple different feature selection methods and the estimation accuracy of soil moisture of different feature selection methods under the optimal number of features.
[0082] Reference Figure 2In order to explore the optimal number of features for soil moisture estimation using different feature selection methods, six feature selection methods were selected, including FRelevant Features using Relief F, Laplacian score (LS), multivariate data analysis (MDA), mean decrease in accuracy, Pearson correlation coefficient (PCC), maximum relevance minimum redundancy (MRMR) and EBSFS. EGRNN was used as the estimation model. According to the feature importance, 1-19 feature variables were selected in turn (see Figure 5 The specific values of characteristic variables) (DOY features are always ranked first) were used for model training (60%, 2635 sets of data) and prediction (40% samples, 1757 sets of data), and a step diagram of the number of characteristic variables with different feature selection methods changing with the accuracy of soil moisture estimation was obtained.
[0083] It can be seen that: (1) with the gradual increase in the number of feature variables, the accuracy of soil moisture estimation gradually improves; (2) the optimal number of features for different feature selection methods is different. Most methods achieve the optimal results when 17-19 features are input. Among them, RReliefF, PCC and MRMR even need to input all 19 features, while the proposed method EBSFS achieves the optimal result when only 14 features are used; (3) when only a few features (such as the first two features) are input, the performance of different methods is also different. Among them, the R values of RReliefF and LS can reach 0.85, the R values of MDA, MRMR and the proposed method EBSFS can be close to 0.9, and the R value of PCC is less than 0.8; (4) The R values of different methods increase at different rates as the number of features increases. Overall, the proposed method EBSFS improves the fastest and can maintain a steady growth, while other methods have an obvious fluctuation trend of "increase first, then decrease, and then increase", indicating that the features selected by these methods have a certain degree of information redundancy.
[0084] Furthermore, the experiment compares the soil moisture estimation results obtained by six feature selection algorithms under the optimal feature variable conditions, as shown in Table 1, Figure 3 and Figure 4As shown in the figure, it can be clearly seen that: (1) the proposed EBSFS feature selection algorithm is significantly better than other comparison methods, with an R increase of about 0.016 and a RMSE reduction of about 0.05; (2) since the optimal number of features for the three feature selection algorithms, RReliefF, PCC and MRMR, is 19, that is, all feature variables are input, their results are consistent; (3) the optimal number of features for the LS and MDA feature selection algorithms is 18 and 17 respectively. Although the number of features is slightly different, the estimation results are also consistent. The reason may be that the feature variables "MODIS_LC_Type1" and "ERA_TP" have little effect on soil moisture estimation, resulting in no change in the results after adding or deleting these two feature variables; (4) the soil moisture map of the proposed method is more detailed in space, and the differences in soil moisture between the eastern, central and western regions are more distinct. At the same time, the scatter plot shows that the estimation results of the proposed method are better fitted with the measured data and are closer to reality.
[0085] Table 1 shows the soil moisture estimation accuracy using different feature selection methods
[0086]
[0087] The above description and accompanying drawings fully illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments.
Claims
1. A sequence feature selection method based on an extended generalized regression neural network, characterized in that: include: Divide the original data set into a training set and a validation set according to a k-fold cross-validation method, and generate k groups of training sets and a cross-validation set corresponding to each group of training sets; wherein the original data set includes multiple groups of original data, and each group of original data includes B feature variables and corresponding actual responses; Initializing a B-dimensional feature subset, wherein the B-dimensional feature subset includes k groups of training sets; Based on the extended generalized regression neural network, the average prediction deviation of all cross-validation sets is calculated according to the k training sets to generate the target loss; Repeatedly search for the feature variables that minimize the target loss and delete them from the B-dimensional feature subset until the target loss no longer decreases, generating the optimal feature subset.
2. The sequence feature selection method based on extended generalized regression neural network according to claim 1, characterized in that: The method of dividing the original data set into a training set and a validation set according to the k-fold cross-validation method, generating k groups of training sets and a cross-validation set corresponding to each group of training sets, includes: Divide the original data set into k training samples on average; For each training sample, use it as a cross-validation set, and use the remaining k-1 training samples among the k training samples as training sets.
3. The sequence feature selection method based on extended generalized regression neural network according to claim 1, characterized in that: The method based on the extended generalized regression neural network calculates the average prediction deviation of all cross-validation sets according to the k training sets to generate the target loss, including: For each training set, the extended generalized regression neural network is trained using the training set to generate an extended generalized regression neural network model of the current training set; According to the characteristic variables in the cross-validation set corresponding to the current training set, the extended generalized regression neural network model based on the current training set predicts the target response; The predicted deviation between the target response and the actual response was calculated based on the preset Pearson correlation coefficient; Calculate the average of all prediction deviations to generate the average prediction deviation, and record the average prediction deviation as the target loss.
4. The sequence feature selection method based on extended generalized regression neural network according to claim 1, characterized in that: The method of repeatedly searching for the feature variables that minimize the target loss and deleting them from the B-dimensional feature subset until the target loss is no longer reduced, thereby generating the optimal feature subset, comprises: Delete certain feature variables from the current feature subset in turn and calculate the target loss; wherein, define the B-dimensional feature subset as the initial current feature subset; When the target loss is reduced, find the feature variable corresponding to the minimum target loss; delete the feature variable from the current feature subset to generate the optimal feature subset; Update the current feature subset to the preferred feature subset and repeat all the above steps until the target loss no longer decreases, and take the final generated preferred feature subset as the optimal feature subset.