Random multi-subspace relief f feature selection method for big data

Abstract

The application provides a ReliefF feature selection method for random multi-subspace of big data, and comprises the following steps: S1, dividing an original feature space to generate a plurality of feature partitions containing a plurality of random subspaces of the same size and no intersection; S2, using a ReliefF or Relief algorithm in each random subspace to obtain a local weight of a feature, and then combining the local weight vectors of the random subspaces in each feature partition to obtain a total weight vector; and S3, integrating the total weight vectors of the plurality of feature partitions into a maximum weight vector of each feature, that is, obtaining the maximum weight vector of all features by averaging the weight scores of the plurality of feature divisions. The application fully considers the diversity of the subspaces and the contribution information of the samples to the features in the feature selection process, and has the ability to explore each subspace in a high-dimensional space.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining methods, and particularly relates to a ReliefF feature selection method for random multi-subspace of big data. BACKGROUND

[0002] With the increasing amount of high-dimensional data generated by different disciplines, great opportunities and challenges have been brought to the research of data mining, knowledge discovery and pattern recognition. Big data usually contains unimportant features, so how to select the optimal feature subset from the feature space has become an important research field. Feature selection (FS) technology, as an important preprocessing method for dimensionality reduction of classification problems, aims to identify a set of relevant and information-rich features from a high-dimensional feature space.

[0003] Existing feature selection methods based on different evaluation criteria are usually divided into three categories, namely filter method, wrapper method and embedded method. The filter method first selects a subset of all features from the data set, and then trains the learner, the feature selection process is independent of the subsequent learning process (such as Relief and ReliefF methods). The wrapper feature selection method directly uses the performance of the model as the evaluation standard of the feature subset (such as the LVW method), so it is superior to the filter method. However, since the learner needs to be trained multiple times during the feature selection process, its cost is usually much larger than that of the filter selection. The embedded feature selection method integrates the feature selection process with the learner training process together, and is completed in the same process (such as the LASSO method).

[0004] When the feature space is high-dimensional, feature selection (FS) is an NP-hard problem, because it needs to find the best subset from 2 d -1 possible subsets on a given data set with d features. This poses a great challenge to the search strategy applied to feature selection. Therefore, most existing feature selection methods identify feature subsets that accurately preserve the structure of the original feature space by estimating the underlying structure of the data set, and these existing feature selection methods lack the ability to explore each subspace in high-dimensional space, and they do not consider the contribution of samples to the feature selection model, i.e. ignoring the diversity of samples. SUMMARY

[0005] The present application aims to at least solve the technical problems existing in the prior art, and particularly innovatively provides a ReliefF feature selection method for random multi-subspace of big data.

[0006] In order to achieve the above purpose of the present application, the present application provides a ReliefF feature selection method for random multi-subspace of big data, comprising:

[0007] S1, divide the original feature space to generate multiple feature partitions containing several non-overlapping random subspaces of the same size;

[0008] S2, in each random subspace, ReliefF or the Relief algorithm is used to obtain the local weights of the features. The local weight vector reflects the local preservation ability of the random subspace and the contribution of the samples to the features. Each random subspace contains multiple features.

[0009] Then, the local weight vectors of the random subspaces in each feature partition are combined to obtain the full weight vector;

[0010] S3 integrates the full weight vectors of multiple feature partitions into the final weight vector of each feature. That is, by averaging the weight scores of multiple feature partitions, the final weight vector of all features is obtained.

[0011] Further, S1 includes:

[0012] The feature partition is represented as:

[0013]

[0014] in, This represents the i-th feature partition;

[0015] P (i,j) express The j-th random subspace in;

[0016] s represents The number of random subspaces in the middle;

[0017] When generating each feature partition, [d / s] features are randomly sampled each time without repetition, until all features have been sampled. When the remaining features are less than [d / s], the remaining features form a random subspace, where [d / s] represents the truncated value of d / s, d represents the total number of features, and s is a hyperparameter.

[0018] Will The j-th random subspace in can be labeled as:

[0019]

[0020] Where f1 (i,j) Represents a random subspace P (i,j) The first feature;

[0021] f2 (i,j) Represents a random subspace P (i,j) The second feature;

[0022] Represents a random subspace P (i,j) The dth ij One feature;

[0023] d ij Represents a random subspace P (i,j) Number of features;

[0024] The set of multiple feature partitions obtained by repeatedly and randomly generating feature partitions can be represented as:

[0025]

[0026] in, This represents the first feature partition generated;

[0027] This represents the second feature partition generated;

[0028] This represents the generated Mth feature partition;

[0029] M represents the number of feature partitions.

[0030] Furthermore, the feature partition contains s random subspaces, and each random subspace has the same number of features [d / s]. If d cannot be expressed as s, then the remaining features form a random subspace.

[0031] Furthermore, the use of ReliefF or the Relief algorithm in each random subspace to obtain the local weights of the features in S2 includes:

[0032] For random subspace P (i,j) Features f in l (i,j) (l=1,2,...,d ij ) Calculate the local feature weights of the feature using ReliefF or the Relief algorithm;

[0033] Then, the random subspace P (i,j) The local weight vector of all features can be represented as:

[0034]

[0035] Where w(f1) (i,j) ) represents the random subspace P (i,j) Local weights of the first feature;

[0036] w(f2 (i,j) ) represents the random subspace P (i,j) Local weights of the second feature;

[0037] Represents a random subspace P (i,j) The dth ij Local weights of each feature;

[0038] d ij Represents a random subspace P (i,j) Number of features;

[0039] (·) T This represents the transpose of a matrix.

[0040] Furthermore, the ReliefF algorithm operates in the random subspace P. (i,j) Obtain feature f l (i,j) (l=1,2,...,d ij The weighting formula for ) is:

[0041]

[0042] Among them, w m (f l (i,j) ) represents the random subspace P (i,j) The local weight of the l-th feature is calculated recursively through m (number of samplings); w m-1 (f l (i,j) ) represents the random subspace P (i,j) Middle feature f l (i,j) The local weights are calculated through m-1 recursive iterations;

[0043] f l (i,j) In the random subspace P (i,j) The l-th feature;

[0044] diff(f l (i,j) ,R,H h ) represents the random subspace P (i,j) Middle feature f l (i,j) Samples R and H h The difference between them;

[0045] diff(f l (i,j) ,R,M h ) represents the random subspace P (i,j) Middle feature f l (i,j) Samples R and M h The difference between them;

[0046] H hrepresents the h-th nearest neighbor of sample R selected from samples with the same class as sample R;

[0047] M h represents the h-th nearest neighbor of sample R selected from samples with different class from sample R;

[0048] m represents the number of sampling times;

[0049] k represents the total number of nearest neighbors;

[0050] Q≠class(R) represents a set of classes different from the class of sample R;

[0051] Q represents the set Q;

[0052] class(R) represents the class of sample R;

[0053] p(·) represents the probability of a specified class over all classes.

[0054] The nearest neighbor is the sample closest to sample R.

[0055] Further, the full weight vector in S2 includes:

[0056] Each feature in the original feature space appears only once in the s random subspaces of the feature partition The full weight vector can be obtained by connecting the local weight vectors of the s subspaces of the feature partition , denoted as:

[0057]

[0058] where represents the full weight vector of the feature partition ;

[0059] w(f1 (i) ) represents the local weight vector of f1 (i) ;

[0060] f1 (i) represents the first feature in the original feature space;

[0061] w(f2 (i) ) represents the local weight vector of f2 (i) ;

[0062] f2 (i) represents the second feature in the original feature space;

[0063] w(f d (i) ) represents the local weight vector of f d (i) ;

[0064] f d (i) denotes the dth feature in the original feature space;

[0065] (·) T denotes the transpose of a matrix.

[0066] Further, the S3 comprises:

[0067] The maximum weight vector is denoted as:

[0068]

[0069] wherein denotes the maximum weight vector;

[0070] denotes the full weight vector of the feature partition ;

[0071] denotes the ith feature partition;

[0072] M denotes the number of feature partitions;

[0073] Then the average value w of the maximum weight vector avg is calculated. i If the feature weight w(f avg ) in the maximum weight vector is greater than w i , the feature is selected as the subset of selected features.

[0074] Further, the method further comprises: taking the accuracy, precision and recall of the KNN classifier and / or the DT classifier as the evaluation index of the method, wherein the classification accuracy of the KNN and DT classifiers is denoted as:

[0075]

[0076] wherein E(f;D) denotes the classification accuracy of the KNN and DT classifiers;

[0077] D denotes the training data set;

[0078] f denotes the KNN classifier or the DT classifier;

[0079] n denotes n-dimensional;

[0080] is an indicator function;

[0081] f(x i ) denotes the prediction result of the classifier in the ith sample;

[0082] c irepresents the true class label of the classifier in the i-th sample.

[0083] In summary, due to the adoption of the above technical solutions, the present application fully considers the diversity of subspaces and the contribution information of samples to features in the feature selection process, and has the ability to explore each subspace in a high-dimensional space.

[0084] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following description, or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0085] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0086] Figure 1 is a radar plot of the average precision, recall, and F1 score of the 26 datasets of the present application under KNN and DT classifiers.

[0087] Figure 2 is a box plot of different datasets under the KNN classifier of the present application.

[0088] Figure 3 is a box plot of different datasets under the DT classifier of the present application.

[0089] Figure 4 is a schematic diagram of the total average number of selected features on the 26 datasets of the present application.

[0090] Figure 5 is a schematic diagram of the total average feature reduction rate on the 26 datasets of the present application. DETAILED DESCRIPTION

[0091] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0092] 1. INTRODUCTION

[0093] In this paper, we use the framework of filter methods for feature selection because they are superior to embedded and wrapper methods in simplicity, scalability, and independence.

[0094] Aiming at the two problems of the background art, this paper proposes a new feature selection method RBEFF. The main idea is to use a large number of random subspaces divided from the original feature space to select a feature subset containing a large amount of information. In the learning process, it is important that each feature should compete fairly in a large number of random subspaces. RBEFF consists of three steps. First, the original feature space is divided into several equal and disjoint feature subspaces, called a feature partition, where the number (size) of the feature partition is M, and the size of each feature partition is d / s. This process is repeated M times. Therefore, in this step, we get M feature partitions containing a large number of subspaces. Second, the local weight of each feature in the subspace is obtained by the following method: the local weight of each feature is calculated in each random subspace using the ReliefF algorithm to form the local weight vector of the random subspace, which reflects the local preservation ability of the random subspace and the contribution ability of the sample to the feature - because the features of the random subspace are not the same, so the k same and different class neighbors of each sample are also not the same, so each feature is assigned a weight of different size in different subspaces. Since the ReliefF algorithm randomly samples m times, the information of different samples is used to measure the weight size of the feature. These local weight vectors in each feature partition constitute the full weight vector, and the full weight vectors in the M feature partitions are integrated to obtain the final weight vector of the feature space. Finally, according to the final weight vector, the feature selection is adaptively performed, that is, without setting the number of selected features, the optimal feature subset is selected to represent all features. A large number of experiments on 26 standard data sets show the effectiveness and effectiveness of the algorithm.

[0095] 2. Background

[0096] 2.1 Related work on feature selection

[0097] The feature estimation problem has attracted extensive attention in the literature. There are several methods to evaluate the quality of features. If the target variable is discrete, the metric values include information gain, Gini coefficient, distance metric, j-measure, Relief, ReliefF, MDL, χ 2 and G statistics. When the target variable is a real-valued function (numeric class and regression problem), mean square, mean absolute error and RReliefF are used. Most of the metrics assume that the features have conditional independence (based on the target variable), so they are not suitable when a large number of feature interactions are designed. Relief, ReliefF and RReliefF algorithms do not make this assumption and can correctly estimate the quality of features in problems with strong dependence between attributes. Therefore, the Relief series algorithm is one of the best preprocessing algorithms so far.

[0098] Relief family of algorithms is a high efficient filter feature selection algorithm. Kira and Rendell first proposed a feature selection algorithm Relief for binary classification problem in 1992. Relief algorithm has been widely used due to its relatively simple, high computational efficiency and satisfactory results. However, its limitation is that it can only deal with two-class data. Therefore, Kononeill analyzed and extended Relief in 1994 to obtain ReliefF algorithm which can be used to deal with noise, incomplete and multi-class problems. In 1997, RReliefF algorithm was generated on how to apply ReliefF to continuous class (regression) problems. After that, many feature selection algorithms based on Relief family of algorithms have been proposed. Relief family of algorithms (Relief, ReliefF and RReliefF) has been widely used to solve practical problems in life, such as signal recognition, transmission line fault location, image processing and image classification, gene classification, pattern recognition, etc. due to its excellent performance.

[0099] 2.2 Relief feature selection algorithm

[0100] Relief algorithm is a feature weighting algorithm. According to the correlation between each feature and class, different weights are given to features. Features with weights less than a certain threshold will be deleted. In Relief algorithm, the correlation between features and classes is based on the ability of features to distinguish between close samples. D is divided into D + ={positive example samples} and D - ={negative example samples}. The algorithm randomly selects a sample R=(f1, f2,..., f d from the training data set D, and then finds the nearest neighbor of R in D + ={positive example samples} Z + , and finds the nearest neighbor of R in D - ={negative example samples} Z - . Then update the weight of each feature according to the following rules: if R is a positive example sample, then near_hit=Z + and near_miss=Z - , on the contrary, near_hit=Z + and near_miss=Z - , where the nearest neighbor is found using Euclidean distance. Then we update the weight of each data using equation (1). Where, when two variables x k and y k are nominal values, diff is calculated by equation (2), when x k and y kWhen the value is numerical, diff is calculated by equation (3). The above process is repeated m times, and finally the average weight of each feature is obtained. The pseudo code of the Relief algorithm is shown in Algorithm 1. The greater the feature weight, the stronger the classification ability of the feature. Conversely, the weaker the classification ability of the feature. The running time of the Relief algorithm increases linearly with the increase of the sampling number m and the original feature number d, so the running efficiency is very high.

[0101] W i i -diff(R i ,near_hit i ) 2 +diff(R i ,near_miss i ) 2 ,(i=1,2,...,d), (1)

[0102] W i denotes the weight of the i-th feature, and its value is calculated by m (the number of sampling times) times of recursion;

[0103] W i denotes the cumulative weight of the i-th feature for m times of sampling, and finally the weight of each feature is averaged by dividing by m.

[0104] diff(R i ,near_hit i ) denotes the difference of the randomly sampled sample R and its nearest neighbor near_hit of the same class at feature i.

[0105] diff(R i ,near_miss i ) denotes the difference of the randomly sampled sample R and its nearest neighbor near_miss of different classes at feature i.

[0106] R i denotes the value of the randomly sampled sample R at feature i.

[0107] near_hit i denotes the value of the nearest neighbor near_hit of the randomly sampled sample R at feature i.

[0108] near_miss i denotes the value of the nearest neighbor near_miss of the randomly sampled sample R at feature i.

[0109]

[0110]

[0111] where v​k It is a standardization unit that standardizes the diff value to the interval [0,1], such as minimum-maximum standardization: for example, the minimum-maximum standardization formula to standardize x to [0,1]. Because x k and v k It is a vector. Maximum refers to the largest value in the vector, and minimum refers to the smallest value in the vector.

[0112]

[0113]

[0114] 2.3 ReliefF Feature Selection Algorithm

[0115] When dealing with multi-class problems, the ReliefF algorithm randomly selects a sample R = (f1, f2, ..., f3) from the training dataset D each time. d Select the k nearest neighbors H of R from samples of the same class as R. h (h = 1, 2, ..., k), and find the k nearest neighbors M of R from samples of different classes from R. h (h = 1, 2, ..., k), where Euclidean distance is used to find the k nearest neighbors. The above process is repeated m times. Then we update the weight of each feature using equation (4), where diff is calculated by equation (5). Thus, we perform feature selection based on the weight of each feature and a given threshold. The pseudocode for the ReliefF algorithm is described in Algorithm 2.

[0116]

[0117] (l=0,1,...,d ij )

[0118] Among them, w m (f l (i,j) ) represents the random subspace P (i,j) The local weight of the l-th feature is calculated recursively through m (number of samplings) times;

[0119] w m-1 (f l (i,j) ) represents the random subspace P (i,j) Middle feature f l (i,j) The local weights are calculated through m-1 recursive iterations;

[0120] f l (i,j) In the random subspace P (i,j) The l-th feature;

[0121] diff(f l (i,j) ,R,H h ) represents the random subspace P (i,j) Middle feature f l (i,j) Samples R and H h The difference between them;

[0122] diff(f l (i,j) ,R,M h ) represents the random subspace P (i,j) Middle feature f l (i,j) Samples R and M h The difference between them;

[0123] H h This indicates selecting the h-th nearest neighbor of sample R from samples of the same class as sample R;

[0124] M h This indicates selecting the h-th nearest neighbor of sample R from samples that have a different class from sample R;

[0125] m represents the number of samples;

[0126] k represents the total number of nearest neighbors;

[0127] Q≠class(R) represents the set of classes that are different from the class of sample R;

[0128] Q represents the set Q;

[0129] class(R) represents the class of sample R;

[0130] p(·) represents the probability of the specified class among all classes.

[0131] The nearest neighbor is the sample that is closest to sample R.

[0132]

[0133] Where, diff(A,R1,R2) represents the difference between samples R1 and R2 under feature A, R1[A] and R2[A] represent the values ​​of samples R1 and R2 under feature A, respectively, and maxA and minA represent the maximum and minimum values ​​of all samples under feature A.

[0134]

[0135] 3. Proposed Algorithm

[0136] 3.1 Symbol Explanation

[0137] Given a classification dataset D = (X, C), represents a data matrix consisting of n samples and d features, where represents the i-th sample (i-th row of matrix X), i.e., x i is d-dimensional (with d features). represents the j-th feature (j-th column of matrix X). represents the class vector. Feature selection refers to selecting a subset of the most effective features from the original features by sufficiently estimating the underlying structure of the data, thereby preserving the structural information of the data. We denote the matrix of this subset as T' = (f1', f2',..., f d′ '), where T' is a submatrix of X T and d' represents the number of selected features. Where T' represents the data matrix composed of selected features, X T represents the transpose of X, d' refers to the number of features selected by feature selection, and d refers to the total number of features in the dataset.

[0138] 3.2 Generation of random subspaces

[0139] The data structure is often hidden in different subspaces of the original feature space. However, it is difficult to estimate the correct feature subspace and the true structural information. Many existing feature selection methods tend to describe the hidden structure of the data by considering global and local information or the minimum redundancy of samples. However, in high-dimensional feature space, the diversity of random subspaces is often ignored, and a large amount of structural information is often hidden. Although some efforts have been made to describe the structural information in the learning subspace, they implicitly assume that there is a single optimal subspace in high-dimensional space, and cannot explore multiple or even a large number of subspaces beyond a single subspace. Therefore, the diversity of subspaces is ignored. To solve this problem, the goal of this paper is to mine the information between feature and class attributes from subspaces. Since it is difficult to find a small number of suitable subspaces, we use random subspace technology and verify that a large number of subspaces can significantly improve the accuracy of feature selection.

[0140] By using a large number of random subspaces, we hope that each feature will compete fairly in the random subspace, where each feature should show the same number in all subspaces, and the generated subspaces should have the same size. Therefore, in this section, we propose a scheme to generate random subspaces. The main idea is to generate multiple feature partitions from the original feature space. Each feature partition is a completely random partition of all features, consisting of a certain number of random subspaces of the same size, so that all features appear the same number of times in the subspaces. Therefore, each feature fairly participates in the fair competition between the generated subspaces.

[0141] Let denotes the i-th feature partition, P (i,j) denotes the j-th random subspace in P

[0142]

[0143] where s denotes the number of random subspaces in P . Since the size of the random subspaces in P should be the same, we let the size of the random subspaces be d / s in this paper. However, in most cases in reality, the value of d / s is not an integer. Therefore, we set the value of d / s by truncation. Thus, when generating each feature partition, we randomly sample [d / s] features each time without repetition until all features have been sampled, and when there are less than [d / s] features left, the remaining features form a random subspace. Where [d / s] denotes the value of d / s after truncation. It should be noted that when d cannot be divided by s, we actually generate s+1 random subspaces. In this paper, the number of subspaces in each feature partition is uniformly denoted as s. Let denote the k-th feature in the random subspace P (i,j) , where d ij denotes the number of features in the random subspace P (i,j) . Therefore, the j-th random subspace in P

[0144]

[0145] Therefore, we can clearly conclude that in a feature partition, each feature appears only once in multiple random subspaces. That is, the different random subspaces divided by a feature partition do not intersect each other. Namely, for a feature space P (i,j) , for

[0146] We repeatedly and randomly generate feature partitions, so the set of multiple feature partitions can be denoted as

[0147]

[0148] where denotes the i-th generated feature partition, and M denotes the number of feature partitions. For a data set, each feature partition contains s random subspaces, and each random subspace has the same number of features. Therefore, ​​There are M-s random subspaces with the same size. According to the generation mechanism of random subspaces, each feature in the original dataset appears M times in the M-s random subspaces.

[0149] 3.3 Weight learning and feature selection for each feature

[0150] In this section, we use random subspaces to describe the underlying information structure of the data and select the informative features that can well preserve the spatial structure. This process is feature selection.

[0151] For the total of M-s random subspaces, the weight of each local feature is obtained according to the ReliefF algorithm (introduced in Section 2.3) in each subspace. Since the features contained in each random subspace are different, the k nearest neighbors of the randomly selected sample are not exactly the same. In the sample space, a sample is randomly drawn under a given random subspace, and the k nearest neighbors of the sample are found to update the weight. Therefore, the underlying structure of the data can be described as much as possible through a large number of random subspaces.

[0152] Given a random subspace P (i,j) , for the features f (i,j) in P l , the local weight of f (i,j) (l = 1, 2,..., d ij ) can be calculated by equation (4) introduced in Section 2.3, w(f (i,j) ) corresponds to W(f i ) of formula (4), i.e. the local feature weight of the lth feature in the jth random subspace of the ith feature partition in the current subspace is calculated by formula (4) (i in formula (4) represents the ith feature in the original feature space). Then, the local weight vector of all d ij features in the random subspace P (i,j) can be expressed as

[0153]

[0154] where w(f l (i,j) ) represents the local weight of the lth feature in P (i,j) . Each feature in the original feature space appears only once in the s random subspaces of the feature partition , and the global weight vector (d is the number of all features) can be obtained by connecting the local weight vectors of its s subspaces, which can be expressed as

[0155]

[0156] where fj (i) (i = 1, 2,..., d) are the features in the original feature space.

[0157] The final weight vector is obtained by taking the average of the full weight vectors of M feature partitions. Then the final weight vector can be represented as

[0158]

[0159] Therefore, the final weight vector of all features can be obtained by coordinating the number of random subspaces. Then the average of the final weight vector is calculated and denoted as w avg For the feature weight in the final weight vector, if w(f i ) is greater than w avg , then the feature is selected as the subset of selected features.

[0160] 3.4 Overall algorithm

[0161] In this section, we will summarize the overall process of the proposed algorithm, whose detailed steps are shown in Algorithm 3.

[0162] The main idea of this algorithm is to generate a large number of random subspaces and obtain the weight of the features in each subspace. Finally, the weight scores of all features are obtained by combining the information of multiple subspaces. In order to ensure that each feature of the original dataset can compete and cooperate fairly in the generated subspace during the process of obtaining each feature weight, a number of random subspaces of the same size and without intersection should be generated in each feature partition. When generating random subspaces, two parameters need to be set, namely the number of feature partitions M contained in each feature partition and the number of random subspaces s. These two parameters are used to adjust the number of feature partitions and the generated random subspaces. We calculate the weight of the features and select the features according to the following content. After generating a large number of random subspaces, the class attribute is added to each random subspace. Then, in each random subspace, the local weight of each feature in the current subspace is obtained using the ReliefF algorithm, and the local weight vector of the subspace is obtained. The local weight vectors of different random subspaces in the feature partition are concatenated to obtain the full weight vector of all features (i.e., the vector obtained from each feature partition). Finally, the final weight vector of all features is obtained by averaging the weight scores of multiple feature partitions (i.e., the final weight vector of each feature is obtained by averaging the different weights assigned to it in multiple feature partitions). In this paper, we use the average of the final weight vector for feature selection, i.e., if the final score of a feature is greater than the average, the feature is selected. Therefore, the final selected feature subset is returned.(as shown in Algorithm 3)

[0163]

[0164]

[0165] 4. Experiments

[0166] We conduct experiments on 26 benchmark datasets with different number of features to demonstrate that the proposed method is indeed superior to several existing feature selection methods.

[0167] 4.1 Datasets

[0168] We obtain 26 datasets from the UCI machine learning repository to verify the effectiveness of the proposed method. It should be noted that the "Ecoli" and "Processed.cleveland" datasets are applied to the algorithm after removing small class samples. Table 1 shows a brief description of the datasets used, including the number of features, the number of samples, and the number of classes.

[0169] Table 1 Datasets

[0170]

[0171]

[0172] 4.2 Compared algorithms

[0173] We compare our method with feature selection methods that use the features used, as well as in the entire feature space or multiple subspaces using different metrics. The details of the comparison algorithms are shown below.

[0174] • Baseline: Classification using all features.

[0175] • Laplace-based feature selection (LS): Feature selection using Laplace scores in the entire feature space.

[0176] • Subspace random collaboration-based feature selection (SRCFS): Feature selection by computing Laplace scores on multiple subspaces.

[0177] • Mutual information-based feature selection (MI): Feature selection using mutual information in the entire feature space.

[0178] • ReliefF-based feature selection (ReliefF): Feature selection using the ReliefF algorithm in the entire feature space.

[0179] • Fuzzy joint mutual information-based feature selection (FJMI): Feature selection by computing fuzzy mutual information.

[0180] 4.3 Evaluation metrics

[0181] The main purpose of feature selection methods is to select fewer features to achieve higher or similar accuracy. In order to verify the superiority of the algorithm, two famous classifiers are used in the experiment: K-Nearest Neighbor (KNN) and Decision Tree (DT). Three standards are used to measure the quality of the algorithm using KNN and DT, namely accuracy, precision, and recall.

[0182] 4.3.1 KNN Classifier

[0183] KNN algorithm was first proposed by Cover and Hart in 1968, and it is one of the simplest classification algorithms and one of the most commonly used classification algorithms. Therefore, it has been widely used in machine learning. For test samples, KNN finds its k-nearest neighbors in the training samples, and then classifies the test samples as the majority class among their k-nearest neighbors. In the KNN algorithm, the number of neighbors k needs to be specified, and the value of the parameter k has a great influence on the classification accuracy of the KNN classifier. Depending on different values, the accuracy can vary greatly. Euclidean distance is commonly used in KNN algorithm, and its definition is formula (12).

[0184]

[0185] where ED represents the Euclidean distance, X1 and represent two samples of d dimensions, and each sample has d features. x 1,i represents the one-dimensional sample X1, x 2,i represents the one-dimensional sample X2.

[0186] 4.3.2 DT Classifier

[0187] Decision Tree (DT) is a simple and commonly used classifier in data mining. The purpose of a decision tree is to learn a model from the given training data and make predictions about the class of test samples. A decision tree is a tree structure where each internal node represents a test of a feature, each branch represents a test output, and each leaf node represents a class. In the DT algorithm, the key to building a decision tree is to choose which feature to use as the basis for classification at the current state. According to different objective functions, there are three main algorithms for building trees: ID3, C4.5, and CART. The main difference is the choice of objective function. ID3 uses information gain, C4.5 uses information gain ratio, and CART uses Gini coefficient. Using different objective functions will have a great impact on classification accuracy. In this paper, we use the ID3 algorithm.

[0188] 4.3.3 Classification Performance

[0189] (1) Classification Accuracy

[0190] The classification accuracy of KNN and DT classifiers is represented as:

[0191]

[0192] where n represents n dimensions, is an indicator function, f(x i ) and c i (i = 1, 2,..., n) represent the prediction results and true class labels of the classifier in n samples, respectively. E(f; D) represents the classification accuracy of the KNN and DT classifiers, and D in E(f; D) represents the training data set, and f represents the KNN classifier or the DT classifier.

[0193] (2) Precision and recall

[0194] For a binary classification problem, samples can be divided into four cases: true positive, false positive, true negative, and false negative according to the combination of their true class and the predicted class of the learner. The binary classification problem can be extended to the multi-classification problem. For the multi-classification problem, one of the classes is regarded as the positive example, and the rest of the classes are regarded as the negative example. Let TP, FP, TN, and FN represent the number of samples corresponding to them, respectively. Then, it is obvious that TP + FP + TN + FN = total number of samples. The confusion matrix of the classification result is shown in Table 2, and the precision and recall are calculated according to the confusion matrix. The definition of the precision and the recall is shown in formulas (14) and (15). The precision of the multi-class problem can be divided into macro-precision and micro-precision, and similarly, the recall of the multi-class problem can also be divided into macro-recall and micro-recall. In this paper, we use the macro-precision and the macro-recall of the multi-classification data set.

[0195] Table 2 Confusion matrix of classification

[0196]

[0197]

[0198]

[0199] 4.3.4 Parameter setting

[0200] In this paper, we use 10-fold cross-validation on our proposed method and other compared methods. The distance metric of KNN uses Euclidean distance and the KNN classifier selects 5 neighbors. Each algorithm is repeated 10 times independently. For the number of random spaces s, we set s to satisfy the size of each random subspace is 6. Note that if the number of all features cannot be divided by 6, then the remainder of division by 6 is the size of the subspace. To compare fairly, the parameters in the compared algorithms and the parameters in the proposed algorithm should be consistent. The values of specific parameters are shown in Table 3.

[0201] Table 3 Parameter settings of the experiment

[0202] Parameter Value(s) r The number of runs 10 K for cross validation 10 k for KNN 5 the distance metric Euclidean k for RBEFF 5 m the sample size for RBEFF 10 M the feature partition 5

[0203] 5. Experimental results

[0204] In this paper, we conduct experiments on 26 standard datasets with different numbers of features to compare our proposed method with several feature selection methods.

[0205] 5.1 Classification accuracy

[0206] Tables 4 and 5 show the average classification accuracy of each feature selection method using KNN and DT classifiers. From Table 4, we can see that the proposed method RBEFF achieves the highest classification on 15 datasets compared with other methods under the KNN classifier, which accounts for more than half of all datasets. From Table 5, we can see that the RBEFF method achieves the highest classification accuracy on 12 datasets compared with other methods under the DT classifier. The average classification accuracy of 26 datasets using KNN and DT classifiers on different algorithms is shown in the last row of Table 4 and Table 5. It can be concluded that our algorithm achieves the highest average accuracy whether using KNN or DT classifier. It can be concluded that our algorithm RBEFF has achieved good results in classification accuracy.

[0207] Table 4 Average classification accuracy of datasets under KNN classifier (black value indicates the best result among several methods)

[0208]

[0209]

[0210] Table 5 Average classification accuracy of datasets under DT classifier (black value indicates the best result among several methods)

[0211] Dataset Baseline ReliefF MI LS SRCFS FJMI RBEFF Data.user 0.9016 0.8401 0.8381 0.9282 0.9256 0.9152 0.8435 Cryotherapy 0.8867 0.8256 0.8733 0.8667 0.8744 0.8567 0.8389 Ecoli 0.9786 0.9705 0.9745 0.9395 0.9745 0.9723 0.9866 Seed1 0.9129 0.8971 0.8595 0.9219 0.9195 0.8314 0.9445 Pima 0.7238 0.5921 0.6780 0.6134 0.6916 0.6864 0.6674 Biopsy 0.9476 0.9558 0.9447 0.9376 0.9407 0.9307 0.9505 Glass 0.7830 0.7000 0.8381 0.8074 0.7888 0.8352 0.7961 Breast-cancer 0.9486 0.9490 0.9453 0.9404 0.9474 0.8486 0.9500 Heart-failure 0.7727 0.7708 0.7602 0.7707 0.7677 0.7910 0.7760 Accent-mfcc 0.7622 0.6972 0.759 0.732 0.7413 0.6222 0.7235 Wine 0.9323 0.926 0.9365 0.9381 0.9006 0.7124 0.9300 Processed.cleveland 0.5152 0.5664 0.4445 0.5003 0.4700 0.4248 0.5983 Diabetes-data-upload 0.9442 0.9115 0.9375 0.9537 0.9440 0.9494 0.9446 Trial 1 1 1 1 1 0.8258 1 Hepatitis 0.5225 0.6515 0.5425 0.5475 0.5977 0.4586 0.7075 Brands 0.632 0.6911 0.6433 0.6331 0.6315 0.6117 0.6873 ForestTypes 0.848 0.7463 0.8396 0.8384 0.8451 0.6111 0.7542 Wdbc 0.7075 0.7292 0.7317 0.7608 0.7492 0.7633 0.7633 Lung-cancer 0.9143 0.9086 0.9218 0.5763 0.9264 0.8868 0.9118 Wpbc 0.8761 0.882 0.8526 0.8484 0.8763 0.8654 0.8912 Ionosphere 0.6984 0.6909 0.6455 0.6761 0.6584 0.5941 0.7471 Qsak 0.8161 0.7767 0.7968 0.8181 0.8049 0.744 0.7679 Sonar 0.7515 0.7471 0.7722 0.781 0.7322 0.71 0.731 Urban land cover 0.7645 0.7584 0.7461 0.742 0.7175 0.5481 0.786 Musk1 0.823 0.7784 0.7979 0.814 0.7707 0.81 0.7655 Clean 1 0.7922 0.7807 0.8059 0.8191 0.7056 0.7788 0.7731 Average 0.73678 0.7897 0.7547 0.7022 0.7588 0.7171 0.8059

[0212] 5.2 Precision

[0213] Table 6 and Table 7 show the average precision of each feature selection method under KNN and DT classifiers. From Table 6, it can be seen that the proposed method RBEFF achieved the highest precision than other algorithms on 11 datasets under KNN classifier. From Table 7, it can also be seen that the proposed method RBEFF achieved the highest precision than other algorithms on 10 datasets under KNN classifier. Therefore, it can be concluded that our proposed method performs better on half of the datasets. The last row of Table 6 and Table 7 gives the overall average precision of 26 datasets using KNN and DT classifiers on different algorithms. It can be seen that our algorithm RBEFF achieved the highest average precision on both KNN and DT classifiers, which indicates that the RBEFF method performs well in terms of precision.

[0214] Table 6 Average precision of datasets under KNN classifier (blackened value indicates the best result among several methods)

[0215]

[0216]

[0217] Table 7 Average precision of datasets under DT classifier (blackened value indicates the best result among several methods)

[0218] Dataset Baseline ReliefF MI LS SRCFS FJMI RBEFF Data.user 0.9154 0.8411 0.8412 0.9233 0.9232 0.9162 0.9387 Cryotherapy 0.8949 0.8515 0.8870 0.8664 0.8725 0.8523 0.8612 Ecoli 0.9606 0.9948 0.9672 0.9662 0.9889 0.9785 0.9989 Seed1 0.9145 0.8979 0.8503 0.9213 0.9147 0.8343 0.9150 Pima 0.7943 0.6997 0.7613 0.7112 0.6538 0.6484 0.7273 Biopsy 0.9363 0.9452 0.9305 0.9149 0.9374 0.9293 0.9275 Glass 0.7718 0.6875 0.8310 0.8017 0.7903 0.8384 0.7801 Breast-cancer 0.9382 0.9470 0.9401 0.9373 0.9448 0.8347 0.9474 Heart-failure 0.6636 0.6579 0.6309 0.6582 0.7307 0.7625 0.6297 Accent-mfcc 0.7763 0.6975 0.7567 0.7283 0.7434 0.6264 0.7221 Wine 0.9400 0.9262 0.9438 0.9362 0.9015 0.7060 0.8600 Processed.cleveland 0.3635 0.3467 0.3168 0.3373 0.3191 0.2390 0.4421 Diabetes-data-upload 0.9771 0.9540 0.9674 0.9643 0.9469 0.9452 0.9774 Trial 1 1 1 1 1 0.8118 1 Hepatitis 0.5930 0.6588 0.6238 0.6158 0.5417 0.4433 0.7006 Brands 0.7099 0.7514 0.7144 0.7111 0.6026 0.5715 0.7400 ForestTypes 0.8426 0.7483 0.8333 0.8331 0.8371 0.5913 0.7500 Wdbc 0.6387 0.6704 0.6525 0.7192 0.6871 0.7033 0.7072 Lung-cancer 0.9351 0.9264 0.9371 0.6458 0.9215 0.8801 0.9375 Wpbc 0.8679 0.8762 0.8484 0.8417 0.8679 0.8525 0.8853 Ionosphere 0.3738 0.3733 0.2617 0.3375 0.5428 0.4534 0.4738 Qsak 0.8627 0.8366 0.8486 0.8640 0.7822 0.7173 0.8202 Sonar 0.7160 0.7332 0.7714 0.7653 0.7337 0.7104 0.7370 Urban land cover 0.7523 0.7618 0.7219 0.7238 0.688 0.5086 0.7860 Musk1 0.803 0.7401 0.7666 0.7864 0.7691 0.8076 0.7319 Clean 1 0.7867 0.7383 0.7782 0.7959 0.7921 0.7746 0.7352 Average 0.7972 0.7793 0.7839 0.7810 0.7859 0.7283 0.7974

[0219] 5.3 Recall

[0220] Table 8 and Table 9 show the average precision of each feature selection method under KNN and DT classifiers. From Table 8, it can be seen that the proposed method RBEFF achieved the highest precision than other algorithms on 10 datasets under KNN classifier. From Table 9, it can also be seen that the proposed method RBEFF achieved the highest precision than other algorithms on 9 datasets under KNN classifier. Therefore, it can be concluded that our proposed method performs better on half of the datasets. The last row of Table 8 and Table 9 gives the overall average precision of 26 datasets using KNN and DT classifiers on different algorithms. It can be seen that our algorithm achieved the highest average recall whether using KNN or DT classifier. Therefore, it can be concluded that our algorithm RBEFF also achieves excellent performance in terms of recall.

[0221] Table 8 Average recall of datasets under KNN classifier (blackened value indicates the best result among several methods)

[0222] Dataset Baseline ReliefF MI LS SRCFS FJMI RBEFF Data.user 0.8480 0.8405 0.8426 0.9238 0.9200 0.8579 0.8448 Cryotherapy 0.7679 0.8118 0.7475 0.7593 0.8244 0.7281 0.8116 Ecoli 0.9703 0.9563 0.9490 0.8734 0.9773 0.9729 0.9726 Seed1 0.8817 0.8958 0.8849 0.878 0.8806 0.8251 0.8958 Pima 0.8344 0.8357 0.8435 0.7743 0.6756 0.6783 0.8238 Biopsy 0.9650 0.9586 0.9595 0.9223 0.9400 0.9252 0.9494 Glass 0.8298 0.8277 0.8609 0.8254 0.7831 0.7682 0.8328 Breast-cancer 0.5474 0.9616 0.9628 0.5478 0.5386 0.5437 0.9775 Heart-failure 0.2060 0.6631 0.2000 0.2018 0.7033 0.7962 0.6567 Accent-mfcc 0.8657 0.642 0.8277 0.7489 0.5386 0.6263 0.7541 Wine 0.6762 0.907 0.6726 0.6784 0.686 0.6371 0.8700 Processed.cleveland 0.2600 0.3551 0.2524 0.2508 0.2483 0.2879 0.4041 Diabetes-data-upload 0.8472 0.8924 0.8624 0.8497 0.8851 0.9371 0.9297 Trial 0.8900 1 0.9556 0.8905 0.9012 0.8366 1 Hepatitis 0.7337 0.9167 0.7382 0.7504 0.5915 0.5430 0.9416 Brands 0.7985 0.8186 0.7961 0.7977 0.5732 0.5820 0.7612 ForestTypes 0.8753 0.8227 0.8748 0.8759 0.8713 0.6502 0.7727 Wdbc 0.6833 0.8021 0.7083 0.7683 0.7071 0.7022 0.8124 Lung-cancer 0.9625 0.8856 0.9563 0.7339 0.9228 0.8945 0.9355 Wpbc 0.7887 0.8028 0.7649 0.798 0.7905 0.8281 0.8312 Ionosphere 0.8424 0.8764 0.8584 0.8433 0.8015 0.6742 0.8315 Sonar 0.7359 0.7941 0.7445 0.683 0.7619 0.7497 0.7000 Urban land cover 0.3481 0.7569 0.3419 0.3484 0.4465 0.3089 0.7751 Musk1 0.9326 0.8754 0.902 0.8567 0.8481 0.8167 0.9222 Clean 1 0.9259 0.8738 0.8801 0.853 0.8477 0.8228 0.9115 Average 0.76066 0.8309 0.7755 0.7373 0.7466 0.7197 0.8367

[0223] Table 9 Average recall of the datasets under DT classifier (blackened values indicate the best result among several methods)

[0224]

[0225]

[0226] Based on the above results, we made radar charts for the average accuracy, recall and precision of 26 datasets under DT and KNN classifiers. From Figure 1 we can see that RBEFF outperforms the comparison methods in terms of the three indicators of different classifiers. Therefore, our conclusion is that our algorithm is competitive and advantageous in most cases in terms of different indicators of different classifiers.

[0227] In this paper, we evaluate the stability of the proposed algorithm according to the boxplot of the average precision on the datasets where the RBEFF method performs well. The boxplot is drawn after the classifier is used on the proposed and comparison algorithms with 10-fold cross-validation and run 10 times, from which the stability of the algorithm can be reflected. In this boxplot, we can see the minimum, maximum, median, first quartile and third quartile of the 10 classification precisions. The line in this boxplot represents the median. Figure 2 and Figure 3 show the boxplots of the average precision of KNN and DT classifiers on different datasets, which can intuitively compare the average performance using different methods. As Figure 2 and Figure 3 show, for most datasets, the proposed RBEFF method has higher boxplots compared to other comparison methods, and our proposed method also has higher precision than other methods. It can also be seen that the height of the box of the proposed algorithm is shorter in most datasets. Therefore, we can conclude that our proposed algorithm achieves better stability.

[0228] 5.5 Selection of the number of features

[0229] As shown in Table 10 and Table 11, it is the total average number of features selected and the reduction rate on 26 datasets using different methods, and the reduction rate refers to the ratio of the number of features reduced after using the feature selection method on the dataset to the total number of features. It can be seen that RBEFF selects the smallest number of features on 14 datasets compared to other algorithms, and also obtains good results on other datasets (ranked second or third). From Table 11, we can also see that our algorithm removes half or more features on 24 datasets. It can be concluded that RBEFF is an effective method to solve the feature selection problem.

[0230] We made bar charts and line charts for the total average and average reduction rate of selected features of 26 datasets using different methods as shown in Figure 4 and Figure 5 It can be seen from Figure 4 that RBEFF achieved the Tatro average size 13.0769, slightly higher than FJMI. However, the method has higher average classification accuracy than FJMI. It can be clearly seen from Figure 5 that RBEFF achieved the highest average reduction rate in the feature selection process compared to other methods.

[0231] Table 10 Number of selected features by different methods on different datasets

[0232]

[0233]

[0234] Table 11 Feature reduction rate by different methods on different datasets

[0235]

[0236]

[0237] 5.6 Wilcoxon signed-rank test

[0238] Finally, the significance difference between the proposed algorithm REBFF and the comparison algorithms is verified by Wilcoxon signed-rank test. Table 12 shows the significance test results of the proposed algorithm and other comparison algorithms on KNN classifier when the significance level a is set to 0.05. From the statistical results (Table 12), it can be seen that all p-values are less than 0.05, which indicates that there is a significant difference between the proposed algorithm and the comparison algorithms, i.e., the proposed algorithm RBEFF is significantly better than other comparison algorithms. The Wilcoxon signed-rank test results prove the effectiveness and superiority of RBEFF.

[0239] Table 12 Wilcoxon signed-rank test of RBEFF method and comparison methods on KNN classifier on 26 datasets (“Yes” means there is a significant difference between the performance of the two methods)

[0240] Pairwise comparision P-value Significant difference RBEFF VS.Baseline 0.0092333 yes RBEFF VS.FJMI 0.0000457 yes RBEFF VS.SRCFS 0.0048130 yes RBEFF VS.LS 0.0007648 yes RBEFF VS.MI 0.0115009 yes RBEFF VS.ReliefF 0.0004456 yes

[0241] 6. Conclusion and future work

[0242] A new method is proposed to deal with feature selection in this paper. The proposed method mines the underlying information of the dataset by randomly sampling the samples and finding their k-nearest neighbors in different subspaces, and uses the distribution samples on the samples to obtain the weights of the features, making full use of the information contained in the samples. We use K-nearest neighbors and decision tree classifiers to verify the performance of our algorithm through 10-fold cross-validation. To study the reliability and quality of the results, we compare our algorithm with commonly used methods such as Baseline, ReliefF, LS, MI, FJMI and SRCFS algorithms on 26 real datasets. The results show that compared with other methods, the algorithm has the highest classification accuracy when the number of selected features is the least or the same. Stability analysis shows that our algorithm RBEFF has an advantage in stability. Finally, the Wilcoxon signed rank test results prove the effectiveness and superiority of RBEFF.

[0243] We can apply our algorithm to ultra-high dimensional or noisy datasets and analyze the impact of different parameters (such as feature division M and random subspace s) on experimental results, which can be further extended in this work.

[0244] The main contributions of this paper can be summarized as follows:

[0245] (1) This paper proposes a RBEFF method that fully considers the diversity of subspaces and the contribution of samples to features in the feature selection process.

[0246] (2) The feature selection framework of RBEFF algorithm can adaptively select effective features.

[0247] (3) A large number of comparisons and statistical analyses are carried out, such as classification accuracy, recall rate, accuracy, bar chart, box plot and Wilcoxon signed rank test.

[0248] (4) RBEFF selects the least number of features in most of the 26 datasets during the feature selection process, and its average reduction rate is also higher than other comparison methods.

[0249] (5) RBEFF algorithm is compared with other algorithms on 26 different size datasets, and the experimental results show that RBEFF algorithm has the highest average accuracy, recall rate and accuracy on KNN and DT classifiers. It can be concluded that RBEFF has superiority and effectiveness in solving feature selection problems.

[0250] Specific embodiments are:

[0251] Embodiment one: in signal recognition, part of the characteristics of the signal is seriously polluted by noise, the ReliefF algorithm is used to evaluate the classification ability of the characteristics, the characteristics with strong classification ability in the signal are extracted, then the similar classification ability redundant characteristics are removed by the characteristic similarity algorithm, and the similar classification ability redundant characteristics are removed by the characteristic similarity algorithm. The remaining strong classification ability characteristics are composed of feature vectors and are classified. The simulation experiment results show that the method can obtain higher correct recognition rate using fewer characteristics. In transmission fault location, the RReliefF algorithm is used to extract a group of candidate features from single voltage measurement data, and the regression elimination algorithm is used to select useful and high line candidate features from the candidate features to accurately obtain the estimation of fault location. This method is not affected by the current signal measurement error in fault location, and does not face the problems and costs related to the transmission and synchronization of two lines. The ReliefF algorithm is one of the most successful feature filtering algorithms in the field of machine learning, and is widely used in feature selection and processing of microarray gene expression data (image classification management, etc.). The signal can be sound, communication signal and radar signal, etc.

[0252] Embodiment two: in image processing and classification, the image data has the characteristics of small sample size and high dimension, and the feature extraction and selection of the image is the most effective method, the purpose of which is to obtain a feature subspace that reflects the essential structure of the data and has higher recognition rate. The RBEFF algorithm mines the underlying structure information of many different random subspaces of high-dimensional image data, extracts the most effective information features describing the image, and thus accurately classifies the image. The RBEFF algorithm can effectively and accurately improve the useful features and improve the classification accuracy of the image. Similarly, in gene classification, in the face of high-dimensional gene expression data with small sample size, RBEFF can automatically select a small number of features (genes) related to diseases and effectively make decision judgments (classification) on gene data.

[0253] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A ReliefF feature selection method for random multi-subspace randomization of high-dimensional image big data, characterized in that, include: S1 involves partitioning the original feature space to generate multiple feature partitions containing several non-overlapping random subspaces of the same size; S1 includes: The feature partition is represented as: ， in, Indicates the first Each feature partition; express The first in A random subspace; express The number of random subspaces in the middle; When generating each feature partition, random sampling is performed each time. [ ] features and do not sample repeatedly until all features have been sampled, and when there are fewer than [ ] features remaining. When ], the remaining features form a random subspace, where [ ]express Truncate the integer value. Indicates the total number of features; It is a hyperparameter; Will The first in A random subspace can be labeled as: ， in Represents a random subspace The first feature; Represents a random subspace The second feature; Represents a random subspace The Middle One feature; Represents a random subspace Number of features; The set of multiple feature partitions obtained by repeatedly and randomly generating feature partitions can be represented as: ， in, This represents the first feature partition generated; This represents the second feature partition generated; Indicates the generated first Each feature partition; M represents the number of feature partitions; S2, the ReliefF algorithm is used in each random subspace to obtain the local weights of the features, and then the local weight vectors of the random subspaces in each feature partition are combined to obtain the full weight vector; S3 integrates the full weight vectors of multiple feature partitions into the final weight vector of each feature. This is achieved by averaging the weight scores of multiple feature partitions to obtain the final weight vector for all features. If the feature weights in the final weight vector are greater than the average, then... If so, then that feature is selected as a subset of the selected features.

2. The ReliefF feature selection method for random multi-subspace in high-dimensional image big data according to claim 1, characterized in that, The feature partition includes There are 10 random subspaces, and each random subspace has the same number of [ ]. If d cannot produce s, then the remainders of the remaining features form a random subspace.

3. The ReliefF feature selection method for random multi-subspace randomization of high-dimensional image big data according to claim 1, characterized in that, The use of the ReliefF algorithm in each random subspace to obtain the local weights of the features in S2 includes: For random subspace Features The local feature weights of the features are calculated using the ReliefF algorithm; Then, the random subspace The local weight vector of all features can be represented as: ， in Represents a random subspace Local weights of the first feature; Represents a random subspace Local weights of the second feature; Represents a random subspace The Middle Local weights of each feature; Represents a random subspace Number of features; This represents the transpose of a matrix.

4. The ReliefF feature selection method for random multi-subspace in high-dimensional image big data according to claim 3, characterized in that, The ReliefF algorithm in random subspace Obtaining features The weighting formula is: ， in, Represents a random subspace Chinese characteristics Local weights, through The value is calculated recursively. Represents a random subspace Chinese characteristics Local weights, through The value is calculated recursively. Indicates in random subspace The Middle One feature; Indicates in random subspace Chinese characteristics Lower sample and The difference between them; Indicates in random subspace Chinese characteristics Lower sample and The difference between them; Indicates from the sample Selecting samples from samples of the same category The The nearest neighbor; Indicates from the sample Selecting samples from samples with different categories The The nearest neighbor; Indicates the number of samples; This represents the total number of nearest neighbors; Representation and Sample A collection of classes with different categories; Represents a set ; Indicates sample Category; This indicates the probability of a specified class among all classes.

5. The ReliefF feature selection method for random multi-subspace in high-dimensional image big data according to claim 1, characterized in that, The full weight vector in S2 includes: By connecting feature partitions To obtain the local weight vector of each subspace The full weight vector is represented as: ， in Representing feature partitions The full weight vector; express The local weight vector; This represents the first feature in the original feature space; express The local weight vector; This represents the second feature in the original feature space; express The local weight vector; Represents the first in the original feature space One feature; This represents the transpose of a matrix.

6. The ReliefF feature selection method for random multi-subspace in high-dimensional image big data according to claim 1, characterized in that, S3 includes: The final weight vector is represented as: ， in This represents the final weight vector; Representing feature partitions The full weight vector; Indicates the first Each feature partition; Indicates the number of feature partitions; Then calculate the final weight vector. average .

Images