A feature selection method based on local distribution difference
By using the ckmeans_1d_dp algorithm and Wasserstein distance to calculate inter-cluster differences, the problem of accurately characterizing the local relationship between features and labels in high-dimensional data is solved, improving the stability and accuracy of feature selection, and making it suitable for modeling complex regression relationships.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING RESEARCH INSTITUTE OF MECHANICAL & ELECTRICAL TECHNOLOGY CO LTD CAM
- Filing Date
- 2025-12-31
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies struggle to accurately capture the local nonlinear or piecewise relationships between features and labels in high-dimensional data modeling, and traditional methods are susceptible to extreme samples, leading to unstable feature weights.
The ckmeans_1d_dp algorithm is used to partition the local distribution patterns of features. The difference between clusters is calculated by combining the Wasserstein distance, and a consistency measure between features and labels is established. The importance of features is measured by the consistency strength, and features with the smallest weight are gradually eliminated.
It achieves accurate characterization of the relationship between non-monotonic or piecewise features and labels, improves the stability and accuracy of feature selection, and is suitable for modeling complex regression relationships.
Smart Images
Figure CN122132791A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of system simulation technology and relates to a feature selection method based on local distribution differences. Background Technology
[0002] The following problems are encountered in the modeling process of performance evaluation based on high-dimensional data: ① High input dimensionality, strong correlation or redundancy among features; ② Non-linear, locally correlated or significantly segmented relationship between features and target variables; ③ Limited sample size and high noise level. Selecting the most relevant and discriminative features or factors from the high-dimensional feature set becomes the primary problem to be solved in the modeling process.
[0003] Traditional methods such as correlation coefficients and mutual information only characterize global linear correlations and struggle to capture local nonlinear or piecewise relationships. Commonly used algorithms based on sample neighborhoods, such as the Relief series, rely on distance metrics and utilize the correspondence between feature differences and label differences to characterize feature importance. However, they still rely on sample-level local neighborhoods rather than the local distribution structure of the feature space. When dealing with nonlinear, piecewise response, or multimodal features, ReliefF estimates feature weights by calculating the average difference of neighborhood samples within the feature space. This global averaging assumption leads to an underestimation of important features, failing to accurately reflect the local association structure between features and labels and lacking the ability to model the local distribution structure of features. Especially when sample distributions are uneven or outliers exist, ReliefF's neighborhood difference metric is easily affected by extreme samples, resulting in large fluctuations and insufficient stability in feature weights. Summary of the Invention
[0004] The purpose of this invention is to at least solve one of the problems existing in the prior art.
[0005] To this end, the present invention provides a feature selection method based on local distribution differences, which is mainly used for regression prediction analysis and modeling that requires screening key features from high-dimensional input data and supporting data-based performance evaluation. It can characterize the local consistency relationship between features and labels based on the segmented statistical distribution law of features, and achieve accurate characterization of complex regression relationships and robust feature selection.
[0006] The technical solution of this invention is as follows:
[0007] A feature selection method based on local distribution differences, the steps of which are as follows:
[0008] Step 1: The ckmeans_1d_dp algorithm is used to automatically divide the one-dimensional value range of each feature, mine the local distribution pattern of the feature according to the feature dimension, and cluster the feature into several clusters according to the local distribution pattern. The empirical distribution of each cluster is estimated.
[0009] Step 2: Use Wasserstein distance to calculate the differences in feature distribution and label distribution among clusters, and establish a consistency measure between features and labels;
[0010] Step 3: Measure the importance of each feature dimension by consistency strength, generate the weight of each feature dimension, and the weight of each feature is equal to the weight of its corresponding feature dimension. After gradually eliminating the features with the smallest weight from smallest to largest, a new feature set is formed, thus realizing feature selection.
[0011] Furthermore, the specific steps of step 1 are as follows:
[0012] Step 1-1: Preprocess and sort the original sample set;
[0013] (1) Let the original sample set be:
[0014] D={(x i ,y i (1) | i = 1, 2, ..., N
[0015] Where, x i =[x i1 ,x i2 ,...,x ij ,...x iM Let x be the M-dimensional feature vector of the i-th sample. ij Let y be the feature of the j-th feature dimension of the i-th sample; i Let be the regression label corresponding to the i-th sample; N is the total number of samples, j = 1, 2, ..., M, where M is the total number of feature dimensions;
[0016] (2) Perform feature standardization on the original sample set:
[0017] Let x be the sequence of all samples with the j-th feature dimension. j =[x 1j ,x 2j ,...,x ij ,...x Nj ], then for x j Each feature x in ij The formulas for linear normalization or standard deviation normalization are as follows:
[0018]
[0019] in, For normalized x ij ;
[0020] (3) Univariate feature extraction and ranking
[0021] For each feature dimension j, extract the corresponding one-dimensional feature sequence, i.e., all the features obtained by formula (2). Form the following formula:
[0022]
[0023] Sort the one-dimensional feature sequences corresponding to all feature dimensions in ascending order, and record the sorting index of their corresponding labels; define the ascending sorted feature sequence and label sequence as follows: and The subscript in the text is the sorting index. This represents the index of feature dimension j, which is currently in ascending order (i), within the original feature sequence. For the label of feature dimension j in ascending order i, π j (i) is the sorted index mapping function. Labels for sorting index mapping functions;
[0024] The sorted dataset is denoted as:
[0025]
[0026] Steps 1-2: Cluster the sorted dataset based on ckmeans_1d_dp to form several clusters;
[0027] (1) Objective and cost function
[0028] Given a sorted dataset, i.e., Equation (4), it is divided into K clusters to minimize the total intra-class squared error SSE, where the index range corresponding to any cluster a in the K clusters is [U a V a The formula for solving SSE is as follows:
[0029]
[0030] In the formula, x(t) is the feature corresponding to sort t, and t is the index interval [U a V a Any value, The mean value within cluster a;
[0031] The calculation formula is as follows:
[0032]
[0033] The index range is calculated as [U a V a The interval cost (price) of U a V a ):
[0034]
[0035] (2) Dynamic programming recursion
[0036] Define D[i, k] as the minimum total cost of dividing the first i samples into k clusters. When i = N and k = K, the expression for D[i, k] is as follows:
[0037] D[i, k] = min m D[m, k - 1] + cost(m + 1, i) (8)
[0038] where m is any value between 0 and i, that is, 0 < m < i, cost(m + 1, i) is the cost of taking points m + 1 to i as a cluster, and D[m, k - 1] is the minimum cost of dividing the first m points into k - 1 clusters;
[0039] Record the backtracking pointer P[i, k] = arg(min m ) to restore the optimal interval partition; when i = N and given k = K, the optimal total cost is D[N, K], and backtracking P gives the interval [U a , V a ;
[0040] Steps 1 - 3, for cluster a in the j-th feature dimension, denoted as C j,a Estimate the empirical distribution:
[0041]
[0042] where is the empirical distribution of cluster a, is the mean of the labels of cluster a, n j,a = V a - U a , t is any value in the index interval [U a , V a ; T is the value range of the original sample set;
[0043] I{x (t) ≤ T} is an indicator function. When x (t) ≤ T, I{x (t) ≤ T} is 1, otherwise it is 0;
[0044] I{y (t) ≤ T} is an indicator function. When y (t) ≤ T, I{y (t) ≤ T} is 1, otherwise it is 0;
[0045] Similarly, the empirical distribution of cluster b in the j-th feature dimension, denoted as C j,b can be calculated and Cluster b can also be any one of the K clusters.
[0046] Furthermore, the boundary conditions of formula (8) are set as follows:
[0047] D[i,1]=cost(1,i), D[0,0]=0, D[i,0]=∞.
[0048] Furthermore, the specific process of step 2 is as follows:
[0049] Calculate the cluster distribution difference vector using Wasserstein distance:
[0050] For the j-th feature dimension, C j,a With C j,b The difference in the characteristic distribution is the one-dimensional Wasserstein distance:
[0051]
[0052] Where W1 represents the one-dimensional Wasserstein distance function;
[0053] Corresponding differences in label distribution:
[0054]
[0055] Construct inter-cluster difference vectors:
[0056]
[0057] Where, m j The number of clusters is the feature dimension j, which can be set according to the number of samples.
[0058] Furthermore, the specific steps of step 3 are as follows:
[0059] Step 3-1, calculate the weight of each feature dimension j:
[0060] After obtaining the inter-cluster difference vector of feature dimension j, i.e., formula (12), the weight of each feature dimension j is calculated according to the following formula:
[0061]
[0062] Where h is any value of H;
[0063] Step 3-2: Normalize the weights of all feature dimensions j:
[0064]
[0065] in, This is the weight vector for all feature dimensions j after normalization.
[0066] Step 3-3: After determining the weights of all feature dimensions j, the weight of each feature is equal to the weight of its corresponding feature dimension, thus obtaining the weights of all features. By excluding the feature with the smallest weight, feature selection is achieved.
[0067] Furthermore, when excluding features, the features with the smallest weights are gradually eliminated according to their weights, from smallest to largest. After excluding features, a new feature set is formed, which is used for regression fitting tasks, and the fitting effect is observed. If the regression effect worsens after eliminating features, it indicates that the feature weight calculation is unreasonable; if the regression effect is still good after eliminating features, it indicates that the feature weight calculation is reasonable.
[0068] Furthermore, when observing the fitting effect, RMSE, MAPE, and R2 methods were used for calculation.
[0069] By applying the above technical solution, the present invention has the following beneficial effects:
[0070] (1) This invention proposes a feature selection idea based on local distribution differences. Based on the idea that "when the differences between features are large, the differences between their corresponding labels are also large", it can effectively capture the complex relationship between non-monotonic or segmented features and labels without relying on global mean or neighborhood average difference measure, thereby avoiding the problem of important features being underestimated.
[0071] (2) This invention uses one-dimensional optimal segmented clustering (ckmeans_1d_dp) to construct feature local clusters and combines it with Wasserstein distance to measure the distribution differences between clusters, which can comprehensively reflect the differences in the distribution patterns of features and labels and improve the accuracy and stability of feature selection.
[0072] (3) This invention proposes a cluster difference consistency scoring method, which directly quantifies the distribution difference between features and labels into feature importance index, realizing a complete automated process from raw data to feature selection, and is suitable for high-dimensional data feature screening in regression fitting tasks. Attached Figure Description
[0073] The accompanying drawings, which form part of this specification, are provided to further illustrate embodiments of the invention and, together with the textual description, explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.
[0074] Figure 1 This is a step diagram of the present invention;
[0075] Figure 2 This is a schematic diagram of the Cook distance fitted after dimensionality reduction using the mutual information method in Example 2;
[0076] Figure 3 This is a schematic diagram of the Cook distance fitted after dimensionality reduction using the RReliefF method in Example 2;
[0077] Figure 4 This is a schematic diagram of the Cook distance fitted after dimensionality reduction using the method in Example 1. Detailed Implementation
[0078] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0079] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0080] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0081] Example 1:
[0082] This embodiment provides a feature selection method based on local distribution differences. See Appendix. Figure 1 The steps of this method are as follows:
[0083] Step 1: The one-dimensional optimal segmented clustering (ckmeans_1d_dp) algorithm is used to automatically divide the one-dimensional value range of each feature, mine the local distribution pattern of the feature based on the feature dimension, avoid the difference in feature dimension weight calculation caused by random sample selection, and improve the stability of feature selection; and cluster the features into several clusters according to the local distribution pattern of the features.
[0084] Step 2: Use Wasserstein distance to calculate the differences in feature distribution and label distribution among clusters, and establish a consistency measure between features and labels;
[0085] Step 3: The importance of each feature dimension is measured by the consistency strength, and the weight of each feature dimension is generated. The weight of each feature is equal to the weight of its corresponding feature dimension, thereby effectively identifying key features with segmented, non-monotonic or multimodal relationships. The larger the weight, the more critical the feature. By gradually eliminating the features with the smallest weight from small to large, the remaining features constitute a new feature set, thus realizing feature selection.
[0086] The specific steps of step 1 are as follows:
[0087] Step 1-1: Preprocess and sort the original sample set:
[0088] (1) Let the original sample set be:
[0089] D={(x i ,y i (1) | i = 1, 2, ..., N
[0090] Where, x i =[x i1 ,x i2 ,...,x ij ,...x iM Let x be the M-dimensional feature vector of the i-th sample. ij Let y be the feature of the j-th feature dimension of the i-th sample; i Let N be the regression label corresponding to the i-th sample; N is the total number of samples, j = 1, 2, ..., M, j > 0, and M is the total number of feature dimensions;
[0091] (2) Perform feature standardization on the original sample set:
[0092] To eliminate the dimensional differences between different feature dimensions, let x be the sequence composed of the j-th feature dimension of all samples. j =[x 1j ,x2j ,...,x ij ,...x Nj ], then for x j Each feature x in ij The formulas for linear normalization or standard deviation normalization are as follows:
[0093]
[0094] in, normalized x ij ;
[0095] (3) Feature extraction and ranking
[0096] For each feature dimension j, extract the corresponding one-dimensional feature sequence, i.e., all the features obtained by formula (2). Form the following formula:
[0097]
[0098] Sort the one-dimensional feature sequences corresponding to all feature dimensions in ascending order (by x). (j) Add subscripts to indicate the sorting sequence number, and record the sorting index of the corresponding label; define the feature sequence and label sequence after ascending order as follows: and The subscript in the text is the sorting index. This represents the index of feature dimension j, which is currently in ascending order (i), within the original feature sequence. For the label of feature dimension j in ascending order i, π j (i) is the sorted index mapping function. Labels for sorting index mapping functions.
[0099] The sorted dataset is denoted as:
[0100]
[0101] Steps 1-2: Cluster the sorted dataset based on ckmeans_1d_dp to form several clusters:
[0102] (1) Objective and cost function
[0103] Given a sorted dataset, i.e., Equation (4), it is divided into K clusters to minimize the total intra-class squared error SSE, where the index range corresponding to any cluster a in the K clusters is [U a V a The formula for solving SSE is as follows:
[0104]
[0105] where \(x(t)\) is the feature corresponding to the sorting order \(t\), and \(t\) is any value in the index range \([U a ,V a ; is the mean within cluster \(a\);
[0106] The calculation formula of
[0107]
[0108] Calculate the interval cost \(cost(U a ,V a ) for the index range \([U a ,V a :
[0109]
[0110] (2) Dynamic programming recurrence
[0111] Define \(D[i,k]\) as the minimum total cost of partitioning the first \(i\) samples into \(k\) clusters. When \(i = N\) and \(k = K\), the expression of \(D[i,k]\) is:
[0112] \(D[i,k]=\min m D[m,k - 1]+cost(m + 1,i)\ (8)
[0113] where \(m\) is any value between \(0\) and \(i\), that is, \(0\lt m\lt i\), \(cost(m + 1,i)\) is the cost of taking points \(m + 1\) to \(i\) as a cluster, and \(D[m,k - 1]\) is the minimum cost of partitioning the first \(m\) points into \(k - 1\) clusters.
[0114] Among them, the boundary conditions of formula (8) are set as:
[0115] \(D[i,1]=cost(1,i), D[0,0]=0, D[i,0]=\infty\);
[0116] Record the backtracking pointer \(P[i,k]=\arg(\min m ) to restore the optimal interval partition; when \(i = N\) and given \(k = K\), the optimal total cost is \(D[N,K]\), and backtracking \(P\) gives the interval \([U a ,V a ;
[0117] Steps 1 - 3, for cluster \(a\) in the \(j\) - th feature dimension, denoted as \(C j,a Perform empirical distribution estimation:
[0118]
[0119] Among them, For the empirical distribution of cluster a, Let n be the mean of the labels of cluster a. j,a =V a -U a t is the index range [U a V a Any value; T is the range of values for the original sample set;
[0120] I{x (t) ≤T} is an indicator function, when x (t) When ≤T, I{x (t) The value is 1 if the value is less than or equal to T, and 0 otherwise.
[0121] I{y (t) ≤T} is an indicator function, when y (t) When ≤T, I{y (t) The value is 1 if the value is less than or equal to T, and 0 otherwise.
[0122] Similarly, the cluster b (denoted as C) of the j-th feature dimension can be calculated. j,b )of and Cluster b can also be any one of the K clusters.
[0123] The specific process of step 2 is as follows:
[0124] Calculate the cluster distribution difference vector using Wasserstein distance:
[0125] For the j-th feature dimension, C j,a With C j,b The difference in the characteristic distribution is the one-dimensional Wasserstein distance:
[0126]
[0127] Where W1 represents the one-dimensional Wasserstein distance function;
[0128] Corresponding differences in label distribution:
[0129]
[0130] Construct inter-cluster difference vectors:
[0131]
[0132] Where, m j The number of clusters is the feature dimension j, which can be set according to the number of samples.
[0133] The specific steps of step 3 are as follows:
[0134] Step 3-1, calculate the weight of each feature dimension j:
[0135] After obtaining the inter-cluster difference vector of feature dimension j, i.e., formula (12), the weight of each feature dimension j is calculated according to the following formula:
[0136]
[0137] Where h is any value of H;
[0138] Step 3-2: Normalize the weights of all feature dimensions j:
[0139]
[0140] in, This is the weight vector for all feature dimensions j after normalization.
[0141] Step 3-3: After determining the weights of all feature dimensions j, the weight of each feature is equal to the weight of its corresponding feature dimension, thus obtaining the weights of all features. By eliminating features with lower weights, the regression prediction accuracy is improved and computational cost is reduced. When eliminating features, features with lower weights are gradually eliminated from smallest to largest according to their weights. After eliminating features with lower weights, a new feature set is formed for the regression fitting task. The RMSE (Root Mean Error), MAPE (Mean Absolute Percentage Error), and R² (Correlation Coefficient) methods are used to calculate and observe the fitting effect. The smaller the RMSE and MAPE values, and the closer the R² (Correlation Coefficient) is to 1, the better the fitting effect.
[0142] When excluding features, features with lower weights are gradually eliminated according to their weights, from smallest to largest. The remaining features form a new feature set, which is used for the regression fitting task, and the fitting effect is observed. If the regression effect deteriorates significantly after eliminating features with low weights, it indicates that the feature weight calculation is unreasonable. If the regression effect remains good after eliminating features with low weights, it indicates that the feature weight calculation is reasonable.
[0143] Example 2:
[0144] Based on Example 1, this embodiment provides a comparison of the effects of applying the method of Example 1, the mutual information method, and the RRelief method to eliminate the features with the lowest weights for a set of high-dimensional data. The comparison results are shown in Table 1.
[0145] Table 1 Comparison of regression prediction results
[0146]
[0147]
[0148] Cook distance is used to assess the influence of a given observation on the model in regression analysis, and it is calculated using a well-known algorithm in the field. This embodiment calculates Cook distances using three different methods (RMSE, MAPE, and R²) to verify their ability to handle outliers. See Appendix. Figure 2-4 It can be seen that in the mutual information and RReliefF methods, the Cook distance of multiple points is relatively large, with a level of about 0.008. However, after dimensionality reduction, the method in Example 1 has a better fitting effect, fewer outliers, and a smaller Cook distance, with a level of about 0.005.
[0149] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.
[0150] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.
[0151] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A feature selection method based on local distribution differences, characterized in that, The steps of this method are as follows: Step 1: Use the ckmeans_1d_dp algorithm to automatically divide the one-dimensional value intervals of each feature, mine the local distribution patterns of the features according to the feature dimensions, cluster them into several clusters according to the local distribution patterns of the features, and estimate the empirical distributions of each cluster. Step 2: Use the Wasserstein distance to calculate the feature distribution differences and label distribution differences between each pair of clusters respectively, and establish the consistency metric of features and labels. Step 3: Measure the importance of each feature dimension through the consistency strength, generate the weights of each feature dimension, and the weight of each feature is equal to the weight of its corresponding feature dimension. After gradually excluding the features with the smallest weights from smallest to largest, a new feature set is formed to achieve feature selection.
2. The feature selection method based on local distribution differences as described in claim 1, characterized in that, The specific steps of Step 1 are as follows: Step 1-1: Preprocess and sort the original sample set. (1) Let the original sample set be: D={(x i ,y i )|i=1,2,…,N} (1) Where, x i =[x i1 ,χ i2 ,...,χ ij ,...χ iM Let x be the M-dimensional feature vector of the i-th sample. ij Let y be the feature of the j-th feature dimension of the i-th sample; i Let be the regression label corresponding to the i-th sample; N is the total number of samples, j = 1, 2, ..., M, where M is the total number of feature dimensions; (2) Perform feature standardization on the original sample set: Let x be the sequence of all samples with the j-th feature dimension. j =[χ 1j ,χ 2j ,...,χ ij ,...χ Nj ], then for χ j Each feature χ in ij The formulas for linear normalization or standard deviation normalization are as follows: in, χ² after normalization ij ; (3) Feature univariate extraction and sorting For each feature dimension j, extract the corresponding one-dimensional feature sequence, i.e., all the features obtained by formula (2). Form the following formula: Sort the one-dimensional feature sequences corresponding to all feature dimensions in ascending order, and record the sorting index of their corresponding labels; define the ascending sorted feature sequence and label sequence as follows: and The subscript in the text is the sorting index. This represents the index of feature dimension j, which is currently in ascending order (i), within the original feature sequence. For the label of feature dimension j in ascending order i, p j (i) is the sorted index mapping function. Labels for sorting index mapping functions; Denote the sorted data set as: Step 1-2: Based on ckmeans_1d_dp, cluster the sorted data set to form several clusters. (1) Objectives and cost functions Given a sorted dataset, i.e., Equation (4), it is divided into K clusters to minimize the total intra-class squared error SSE, where the index range corresponding to any cluster a in the K clusters is [U a V a The formula for solving SSE is as follows: In the formula, x(t) is the feature corresponding to sort t, and t is the index interval [U a V a Any value, The mean value within cluster a; The calculation formula is as follows: The calculated index range is [U a V a The interval cost (price) of U a V a ): (2) Dynamic programming recursion Define D[i,k] as the minimum total cost of dividing the first i samples into k clusters. When i = N and k = K, the expression of D[i,k] is: D[i,k]=min m D[m,k-1]+cost(m+1,i) (8) where m is any value between 0 and i, that is, 0 < m < i, cost(m + 1,i) is the cost of taking points m + 1 to i as a cluster, and D[m,k - 1] is the minimum cost of dividing the first m points into k - 1 clusters; Record the backtracking pointer P[i,k] = arg(min) m To recover the optimal interval partition; when i = N and given k = K, the optimal total cost is D[N,K], and backtracking P yields the interval [U... a V a ]; Steps 1-3: For cluster a with the j-th feature dimension, denoted as C j,a Perform empirical distribution estimation: in, For the empirical distribution of cluster a, Let n be the mean of the labels of cluster a. j,a =V a -U a t is the index range [U a V a Any value; T is the range of values for the original sample set; I(x (t) ≤T} is an indicator function, when x (t) When ≤T, I(x) (t) The value is 1 if the value is less than or equal to T, and 0 otherwise. I{y (t) ≤T} is an indicator function, when y (t) When ≤T, I{y (t) The value is 1 if the value is less than or equal to T, and 0 otherwise. Similarly, the cluster b of the j-th feature dimension can be calculated, denoted as C. j,b of and Cluster b can also be any one of the K clusters.
3. The feature selection method based on local distribution differences as described in claim 2, characterized in that, Set the boundary conditions of formula (8) as: D[i,1] = cost(1,i), D[0,0] = 0, D[i,0] = ∞.
4. The feature selection method based on local distribution differences as described in claim 2, characterized in that, The specific process of Step 2 is as follows: Use the Wasserstein distance to calculate the cluster class distribution difference vector: For the j-th feature dimension, C j,a With C j,b The difference in the characteristic distribution is the one-dimensional Wasserstein distance: where W1 represents the one-dimensional Wasserstein distance function; The corresponding label distribution difference: Construct the inter-cluster difference vector: Where, m j The number of clusters is the feature dimension j, which can be set according to the number of samples.
5. The feature selection method based on local distribution differences as described in claim 4, characterized in that, The specific steps of Step 3 are as follows: Step 3-1: Calculate the weight of each feature dimension j: After obtaining the inter-cluster difference vector of feature dimension j, that is, formula (12), calculate the weight of each feature dimension j according to the following formula: where h is any value of H; Step 3-2: Normalize the weights of all feature dimensions j: in, This is the weight vector for all feature dimensions j after normalization. Step 3-3: After determining the weights of all feature dimensions j, the weight of each feature is equal to the weight of its corresponding feature dimension, and then the weights of all features can be obtained. By excluding the features with the smallest weights, feature selection is achieved.
6. The feature selection method based on local distribution differences as described in claim 5, characterized in that, When excluding features, according to the weights, gradually eliminate the features with the smallest weights from smallest to largest. After excluding the features, a new feature set is formed for the regression fitting task, and the fitting effect is observed. If the regression effect deteriorates after eliminating the features, it indicates that the feature weight calculation is unreasonable; if the regression effect is still good after eliminating the features, it indicates that the feature weight calculation is reasonable.
7. The feature selection method based on local distribution differences as described in claim 6, characterized in that, When observing the fitting effect, use the RMSE, MAPE, and R2 methods to calculate respectively.