A feature selection method, device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a feature selection method and device, electronic equipment and storage medium. The method comprises: obtaining a training sample set, calculating the conditional entropy corresponding to each dimension of features in a feature set composed of the same dimension features of multiple samples, and calculating the selected probability corresponding to the features of the dimension based on the conditional entropy; initializing a preset number of particles based on the selected probability corresponding to the features of each dimension, for each element in each particle, if the value of the element is greater than a preset feature threshold, determining that the feature corresponding to the element in the particle is selected, obtaining a selected feature subset corresponding to the particle; determining the value of the objective function based on the selected feature subset corresponding to each particle, if the objective function converges, determining the selected features of each sample as target features; and if the objective function does not converge, adaptively updating each particle. By using the method, the convergence speed of the machine learning algorithm is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine learning, and particularly relates to a feature selection method and device, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, machine learning technology has been greatly developed and applied in many fields. For example, image classification is achieved by processing image data using a machine learning algorithm. However, in the context of big data, the machine learning algorithm needs to process a huge amount of data, and the data features present high-dimensional characteristics. There is a data redundancy problem in the high-dimensional data. Data redundancy can cause the processing time of the machine learning algorithm to be longer, the complexity of data processing to be increased, and the accuracy of the machine learning algorithm to be affected. Therefore, it is necessary to perform feature selection on high-dimensional data to reduce the data dimension. Feature selection refers to selecting a group of representative data features according to a certain standard to remove data redundancy and reduce the complexity of data processing.

[0003] At present, existing feature selection methods include filter methods, wrapper methods and embedded methods. The filter method selects features based on an unsupervised strategy using information theory and fuzzy set indicators. The wrapper method selects features in combination with classification performance indicators. The embedded method embeds feature selection in the training process of a classifier, and the quality of the selected features depends on the degree of excellence of the trained classifier. However, the existing feature selection methods lack self-adaptive ability, which limits the convergence speed of the machine learning algorithm. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a feature selection method, device, electronic device and storage medium to improve the convergence speed of the machine learning algorithm.

[0005] To achieve the above purpose, the embodiments of the present application provide a feature selection method, comprising:

[0006] obtaining a training sample set, wherein the training sample set comprises a plurality of samples, and each sample comprises D-dimensional features;

[0007] calculating the conditional entropy corresponding to the features of each dimension in a feature set composed of the same dimension features of the plurality of samples, and calculating the selected probability corresponding to the features of each dimension based on the conditional entropy;

[0008] initializing a preset number of particles by a particle swarm optimization algorithm based on the selected probability corresponding to the features of each dimension, wherein each particle corresponds to a decision variable, and the decision variable comprises D elements, each element corresponding to the features of a dimension;

[0009] For each element in each particle, if the value of the element is greater than a preset feature threshold, it is determined that the feature corresponding to the element in the particle is selected, and a selected feature subset corresponding to the particle is obtained;

[0010] A value of a target function is determined based on the selected feature subset corresponding to each particle, and the value of the target function is used to represent a feature selection ratio and a feature classification error rate of each sample;

[0011] It is determined whether the target function converges based on the value of the target function;

[0012] If the target function converges, the selected feature of each sample is determined as a target feature;

[0013] If the target function does not converge, each particle is updated, and the step of determining, for each element in each particle, if the value of the element is greater than a preset feature threshold, that the feature corresponding to the element in the particle is selected is returned.

[0014] Optionally, the condition entropy corresponding to the feature of the dimension is calculated by:

[0015] The condition entropy corresponding to the feature of the dimension is calculated by the following formula:

[0016]

[0017] Wherein, L is a class set to which samples in a training sample set belong, F m is a feature set composed of the mth feature of a plurality of samples in the training sample set, H(L|F m ) is a condition entropy corresponding to the feature Fm of the dimension, represents a sub-event of F m , F m is a tth clustering result obtained by a clustering algorithm, and l is a sub-event of L, is a joint probability, represents a probability that a sample in the training sample set belongs to the sub-event and the lth class, is a conditional probability, represents a probability that the sample belongs to the lth class under the condition of the sub-event .

[0018] Optionally, the selected probability corresponding to the feature of the dimension is calculated based on the condition entropy by:

[0019] The selected probability corresponding to the feature of the dimension is calculated based on the condition entropy by the following formula:

[0020]

[0021] Among them, P(L|F m ) represents the feature F of this dimension. m The corresponding selection probability, where L is the set of categories to which the samples in the training sample set belong, and F... m H(L|F) is the feature set consisting of the m-th dimension features of multiple samples in the training sample set. m ) represents the conditional entropy corresponding to the feature of this dimension, and D represents the number of feature dimensions included in the sample.

[0022] Optionally, the initialization of a preset number of particles based on the selection probabilities corresponding to features in each dimension using a particle swarm optimization algorithm includes:

[0023] Set a preset number of particles corresponding to decision variables containing D elements;

[0024] For each element in each particle, a uniform random number is generated. If the uniform random number is less than the selection probability of the feature corresponding to the element, the element of the particle is assigned a value greater than a preset feature threshold and less than or equal to 1. Otherwise, the element of the particle is assigned a value greater than zero and less than the preset feature threshold. A preset number of initialized particles are obtained.

[0025] Optionally, the objective function can be represented by the following formula:

[0026]

[0027] Where X = {x1, x2, ..., x} j ,…,x D} are decision variables. The proportion of features chosen for decision variable X, x j Let X be the j-th element of the decision variable X, threshold be the preset feature threshold, D be the dimension of the features included in the sample, and N be the number of samples in the training sample set. a represents the classification error rate when using the selected feature set of decision variable X for classification. k Let a represent the k-th sample in the training sample set. k (X) represents sample a using the features selected by decision variable X. k , l k For a k The true category identifier, f(a) k (X) represents sample a k (X) The class label obtained after classification, if sample a k If (X) the classification result is the same as the true category label result, then δ(f(a) k (X)),l k ) = 0, otherwise δ(f(a) k(X)),l k ) = 1.

[0028] Optionally, the updating each particle comprises:

[0029] determining current speed information and current position information of each particle;

[0030] determining the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle, to obtain the updated position information of the particle.

[0031] Optionally, the determining the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle comprises:

[0032] The speed of each element of the particle at the next moment is determined based on the current speed information and the current position information of each particle by using the following formula:

[0033]

[0034] w = 0.9 - 0.5exp(-r) ;

[0035]

[0036] wherein, H (L|F m ) is the conditional entropy corresponding to the feature F m , threshold is a preset feature threshold, x m is the mth element of the current particle, t is the number of iterations, V i (t+1) is the speed of the ith particle at the t+1 generation, r1 and r2 are random numbers between 0 and 1, V i (t) is the current speed of the ith particle, X i (t) is the current position information of the ith particle, w is the inertia weight, r represents the average uncertainty of the selected feature of the current solution, P i best (t) is the current individual optimal solution of the ith particle, G i best (t) is the global optimal solution, w*V i (t) indicates the sub-particle is close to the global optimal solution; the higher the quality of the solution, the smaller the value of r and w, and the greater the probability of the ith particle performing local search around the current solution; on the contrary, the worse the quality of the solution, the greater the value of w, and the greater the probability of the ith particle performing global search.

[0037] In order to achieve the above purpose, the embodiment of the present application further provides a feature selection device, comprising:

[0038] a sample obtaining module configured to obtain a training sample set, wherein the training sample set comprises a plurality of samples, and each of the samples comprises D-dimensional features;

[0039] a calculating module configured to calculate, for each dimension of a feature set composed of the same dimension features of the plurality of samples, a conditional entropy corresponding to the features of the dimension, and calculate a selected probability corresponding to the features of the dimension based on the conditional entropy;

[0040] an initializing module configured to initialize a preset number of particles by a particle swarm optimization algorithm based on the selected probabilities corresponding to the features of the dimensions, wherein each of the particles corresponds to a decision variable, and the decision variable comprises D elements, each of which corresponds to a feature of a dimension;

[0041] a feature selecting module configured to, for each element in each particle, if a value of the element is greater than a preset feature threshold, determine that a feature corresponding to the element in the particle is selected, and obtain a selected feature subset corresponding to the particle;

[0042] a target function value determining module configured to determine a value of a target function based on the selected feature subsets corresponding to the particles, wherein the value of the target function is used to represent a feature selection ratio and a feature classification error rate of each sample;

[0043] a target feature determining module configured to determine whether the target function converges based on the value of the target function, and if the target function converges, determine selected features of each sample as target features, and if the target function does not converge, update the particles and return the step of, for each element in each particle, if a value of the element is greater than a preset feature threshold, determining that a feature corresponding to the element in the particle is selected.

[0044] Optionally, the calculating module is specifically configured to calculate the conditional entropy corresponding to the features of the dimension by using the following formula:

[0045]

[0046] wherein L is a class set to which samples included in the training sample set belong, F m is a feature set composed of the mth-dimensional features of the plurality of samples in the training sample set, H(L|F m ) is a conditional entropy corresponding to the features Fm of the dimension, represents a sub-event of F m , F m is a tth-class clustering result obtained by a clustering algorithm, and l is a sub-event of L, is a joint probability, represents a probability that a sample in the training sample set belongs to both the sub-event and the lth class, is a conditional probability, represents the probability that a sample belongs to class under the condition that

[0047] Optionally, the calculation module is specifically configured to calculate the selected probability corresponding to the feature of the dimension based on the conditional entropy by using the following formula:

[0048]

[0049] wherein, P(L|F m ) is the selected probability corresponding to the feature F m of the dimension, L is a set of classes to which samples in the training sample set belong, F m is a feature set composed of the mth feature of the plurality of samples in the training sample set, H(L|F m ) is the conditional entropy corresponding to the feature of the dimension, and D is the feature dimension included by the sample.

[0050] Optionally, the initialization module is specifically configured to set a preset number of particles corresponding to decision variables containing D elements; for each element in each particle, a uniform random number is generated, if the uniform random number is less than the selected probability of the feature corresponding to the element, a value greater than a preset feature threshold and less than or equal to 1 is assigned to the element of the particle, otherwise, a value greater than zero and less than the preset feature threshold is assigned to the element of the particle; and a preset number of initialized particles are obtained.

[0051] Optionally, the objective function is represented by using the following formula:

[0052]

[0053] wherein, X={x1,x2,…,x j ,…,x D} is a decision variable, p is the selected feature proportion of the decision variable X, x j is the jth element of the decision variable X, threshold is a preset feature threshold, D is the dimension of the feature included by the sample, N is the number of samples in the training sample set, represents the classification error rate of classification by using the selected feature set of the decision variable X, a k represents the kth sample in the training sample set, a k (X) represents the sample a k using the selected feature of the decision variable X, l k is the true class identifier of a k , f(a k (X)) represents the class label obtained after the sample a k (X) is classified, and if the sample ak (X) the result after classification is same as the true class label result, then delta (f(a k (X)), l k ) = 0, otherwise delta (f(a k (X)), l k ) = 1.

[0054] Optionally, the target feature determination module is specifically configured to determine current speed information and current position information of each particle; and determine the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle, to obtain updated position information of the particle.

[0055] Optionally, the target feature determination module is specifically configured to determine the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle, by using the following formula:

[0056]

[0057] w = 0.9 - 0.5exp(-r);

[0058]

[0059] wherein H(L|F m ) is the conditional entropy corresponding to the feature F m , threshold is a preset feature threshold, x m is the mth element of the current particle, t is the number of iterations, V i (t+1) is the speed of the ith particle at the t+1 generation, r1 and r2 are random numbers between 0 and 1, V i (t) is the current speed of the ith particle, X i (t) is the current position information of the ith particle, w is the inertia weight, r represents the average uncertainty of the selected feature of the current solution, P i best (t) is the current individual optimal solution of the ith particle, G i best (t) is the global optimal solution, w*V i (t) indicates the particle is close to the global optimal solution; the higher the quality of the solution, the smaller the value of r and w, and the greater the probability of the ith particle performing local search around the current solution; on the contrary, the worse the quality of the solution, the greater the value of w, and the greater the probability of the ith particle performing global search.

[0060] To achieve the above object, the embodiment of the present application further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus.

[0061] The memory is used for storing a computer program.

[0062] The processor is used for executing the program stored on the memory, and realizes the feature selection method steps.

[0063] To achieve the above object, the embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the feature selection method steps.

[0064] To achieve the above object, the embodiment of the present application further provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the feature selection method steps.

[0065] The embodiment of the present application has the following beneficial effects:

[0066] By using the method provided by the embodiment of the present application, a training sample set is obtained, the conditional entropy corresponding to the feature of each dimension in the feature set composed of the same dimension features of multiple samples is calculated, and the selected probability corresponding to the feature of each dimension is calculated based on the conditional entropy; based on the selected probabilities corresponding to the feature sets, a preset number of particles are initialized to optimize the decision variables, for each element of the decision variable corresponding to each particle, if the value of the element is greater than a preset feature threshold, it is determined that the feature corresponding to the element in the decision variable is selected, and a group of selected feature subsets corresponding to the decision variable is obtained; the target function value is determined based on the feature subsets corresponding to the decision variables, if the target function converges, the selected features of each sample are determined as target features; if the target function does not converge, the decision variables are updated, and the step of determining that the feature corresponding to each element in the decision variable is selected is returned for each element of the decision variable corresponding to each particle, if the value of the element is greater than a preset feature threshold. The method provided by the embodiment of the present application can calculate the selected probability corresponding to the feature of each dimension based on the conditional entropy corresponding to the feature of each dimension, that is, initialize the particles based on the conditional entropy, utilize the prior information of each feature of the sample, reduce the probability of blind search, improve the initialization quality, and further improve the convergence speed of the machine learning algorithm.

[0067] Of course, implementing any product or method of the present application does not necessarily require achieving all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0069] Figure 1 A flowchart of a feature selection method provided in an embodiment of the present invention;

[0070] Figure 2 A schematic diagram of a feature selection device provided in an embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0073] Figure 1 A flowchart of a feature selection method provided in an embodiment of the present invention, such as... Figure 1 As shown, it includes the following steps:

[0074] Step 101: Obtain the training sample set.

[0075] The training sample set consists of multiple samples, each containing D-dimensional features. For example, the training sample set A = {a1, a2, ..., a...} N} T ∈R N*D Where a1, a2, ..., a N These are the samples in the training sample set, a1, a2, ..., a... N All include D-dimensional features. A sample can specifically be a sample image to be classified, and the D-dimensional features included in the sample can be the D-dimensional features of the image to be classified.

[0076] Step 102: For each dimension of the feature set consisting of the same dimension of features of multiple samples, calculate the conditional entropy corresponding to the feature of that dimension, and calculate the selection probability corresponding to the feature of that dimension based on the conditional entropy.

[0077] In this embodiment of the invention, features of the same dimension from multiple samples in the training sample set can be defined as a feature set, for example, samples a1, a2, ..., a N The set of features of dimension s can be used as the feature set corresponding to the feature of dimension s, where 1≤s≤D.

[0078] In one possible implementation, the conditional entropy corresponding to the feature in each dimension can be calculated using the following formula:

[0079]

[0080] Where L is the set of categories to which the samples in the training sample set belong, and F m H(L|F) is the feature set consisting of the m-th dimension features of multiple samples in the training sample set. m ) represents the feature F of this dimension. m The corresponding conditional entropy, For F m The sub-event of F represents F m The clustering result of the t-th cluster obtained by the clustering algorithm, where l is a sub-event of L, For joint probability, This indicates that a sample in the training sample set simultaneously belongs to a sub-event. The probability of class l, For conditional probability, Indicates in sub-event Given the condition, the probability that a sample belongs to class l.

[0081] In this embodiment of the invention, H(L|F m The known features F were measured. m The uncertainty of correctly classifying the training sample set A, H(L|F) m The larger the conditional entropy, the less discriminative information is obtained from the m-th dimension feature. Therefore, conditional entropy can be used as a criterion for whether to select the m-th dimension feature.

[0082] In one possible implementation of this invention, the selection probability corresponding to each dimension's feature can be calculated based on the conditional entropy, including:

[0083] The selection probability of the feature in this dimension is calculated based on the conditional entropy using the following formula:

[0084]

[0085] Among them, P(L|F m ) represents the feature F of this dimension. m The corresponding selection probability, where L is the set of categories to which the samples in the training sample set belong, and F... mH(L|F m ) is the conditional entropy corresponding to the dimension of the feature, and D is the feature dimension included in the sample.

[0086] The selected probability corresponding to the dimension of the feature is used to represent the probability of the dimension of the feature being selected as a discriminant feature.

[0087] In the embodiment of the present application, the greater the value of P(L|F m ), the higher the probability of selecting the feature F m . By calculating P(L|F m ), the blind search in the random initialization process is reduced.

[0088] In step 103, based on the selected probability corresponding to the dimension of the feature, a preset number of particles are initialized by a particle swarm optimization algorithm.

[0089] Each particle corresponds to a decision variable including D elements, and each element corresponds to a dimension of the feature.

[0090] In one possible implementation, the preset number of decision variables can be initialized by the following steps A1-A2:

[0091] Step A1: setting a particle corresponding to the preset number of decision variables including D elements;

[0092] Step A2: for each element in each particle, generating a uniform random number, if the uniform random number is less than the selected probability of the feature corresponding to the element, assigning a value greater than the preset feature threshold and less than or equal to 1 to the element of the particle, otherwise, assigning a value greater than zero and less than the preset feature threshold to the element of the particle; obtaining the preset number of initialized particles.

[0093] Specifically, a real number coding method can be used to set the particle X={x1,...,x i ,...,x D} of the decision variable including D zero-value elements, x i ∈[0,1], wherein x j corresponds to the feature set composed of the jth dimension of the feature of each sample in the training sample set, and threshold is a preset feature threshold, x j > threshold indicates that the jth dimension of the feature of each sample in the training sample set is selected.

[0094] The preset number can be set according to actual application requirements, for example, 5 or 10, which is not limited here.

[0095] In this embodiment of the invention, a predetermined number of particles X = {x1, ..., x2}, each containing D elements, are initialized. i , ..., x D}, where x i The rules for determining the value are as follows:

[0096] Generate a uniformly random number r between 0 and 1, when r < P(L|F m ), x i Setting the threshold to a random number between 1 and 0 indicates that the feature is selected; otherwise, setting it to a random number between 0 and 1 indicates that the feature is not selected. Based on P(L|F m This initialization particle fully utilizes the prior information of each feature of each sample in the training sample set, P(L|F m The larger the ), the better to choose F. m The higher the probability, the lower the probability of blind searching and the higher the initialization quality.

[0097] Step 104: For each element in the decision variable corresponding to each particle, if the value of the element is greater than the preset feature threshold, it is determined that the feature corresponding to the element in the particle is selected, and a set of feature subsets corresponding to the particle is obtained.

[0098] For example, in decision variable X, when x2 > threshold, it means that the feature set consisting of the second dimension features of each sample in the training sample set is selected, that is, the second dimension features of each sample in the training sample set are selected.

[0099] Step 105: Determine the value of the objective function based on the selected feature subset corresponding to each particle.

[0100] The value of the objective function is used to characterize the feature selection ratio and feature classification error rate for each sample.

[0101] Specifically, in this embodiment of the invention, the objective function can be represented by the following formula:

[0102]

[0103] Where X = {x1, x2, ..., x} j ,…,x D} are decision variables. The proportion of features chosen for decision variable X, x j Let X be the j-th element of the decision variable X, threshold be the preset feature threshold, D be the dimension of the features included in the sample, and N be the number of samples in the training sample set. a represents the classification error rate when using the selected feature set of decision variable X for classification. kdenotes the kth sample in the training sample set, a k (X) denotes the sample a k adopting the selected feature of the decision variable X k k the true class label of a k (X) denotes the sample a k (X) after classification, if the classification result of the sample a k (X) is same as the true class label result, then δ(f(a k (X)), l k ) = 0, otherwise δ(f(a k (X)), l k ) = 1.

[0104] In the embodiment of the present application, the sample a k after feature selection can be predicted and classified to obtain the class label f(a k (X) of the predicted classification, and then whether the class label f(a k (X) of the predicted classification of the sample a k is consistent with the true class label l k of the sample a k is determined to determine whether the predicted classification of the sample a k is correct, and the feature classification error rate δ is obtained, if the classification result of the sample a k is same as the true class label result, then δ(f(a i (X)), l i ) = 0, otherwise δ(f(a i (X)), l i ) = 1.

[0105] Step 106, determining whether the objective function converges based on the value of the objective function.

[0106] Step 107, if the objective function converges, determining the selected feature of the training sample as the target feature.

[0107] Specifically, any existing inflection point algorithm can be used to determine the target feature of each sample. For example, the following steps B1-B4 can be used to determine the target feature:

[0108] Step B1: smoothing the current non-dominated solution by using B-spline interpolation method;

[0109] Step B2: uniformly sampling on the processed curve;

[0110] Step B3: calculating the curvature change of each sampling point;

[0111] ​Step B4: the inflection point is the nearest solution of the sampling point with the largest curvature change, and the feature corresponding to the inflection point is determined as the target feature.

[0112] Step 108: if the objective function does not converge, update each particle and return to perform step 104.

[0113] By using the method provided in the embodiment of the present application, a training sample set is obtained, the conditional entropy corresponding to each dimension feature in each feature set composed of the same dimension features of multiple samples is calculated, and the selected probability corresponding to the feature of each dimension is calculated based on the conditional entropy; based on the selected probability corresponding to each feature set, a preset number of particles are initialized to optimize the decision variable, for each element of the decision variable corresponding to each particle, if the value of the element is greater than a preset feature threshold, it is determined that the feature corresponding to the element in the decision variable is selected, and a group of selected feature subsets corresponding to the decision variable is obtained; the target function value is determined based on the feature subset corresponding to each decision variable, if the objective function converges, the selected feature of each sample is determined as the target feature; if the objective function does not converge, each decision variable is updated, and the step of determining that the feature corresponding to each element in the decision variable is selected if the value of the element is greater than a preset feature threshold is returned for each element of the decision variable corresponding to each particle. The method provided in the embodiment of the present application can calculate the selected probability corresponding to the feature of each dimension based on the conditional entropy corresponding to the feature of each dimension, that is, initialize the particle based on the conditional entropy, utilize the prior information of each feature of the sample, reduce the probability of blind search, improve the initialization quality, and further improve the convergence speed of the machine learning algorithm.

[0114] In a possible implementation, the step of updating each decision variable can specifically include the following steps C1-C2:

[0115] Step C1: determine the current speed information and the current position information of each particle;

[0116] Step C2: based on the current speed information and the current position information of each particle, determine the speed of each element of the particle at the next moment to obtain the updated position information of the particle.

[0117] Specifically, the speed of each element of the particle at the next moment can be determined based on the current speed information and the current position information of each particle by using the following formula in this step:

[0118]

[0119] w = 0.9 - 0.5exp(-r);

[0120]

[0121] where H(L|F m ) is the conditional entropy of the feature F m , threshold is a preset feature threshold, x m is the mth element of the current particle, t is the number of iterations, V i (t+1) is the velocity of the ith particle in the t+1 generation, r1 and r2 are random numbers between 0 and 1, V i (t) is the current velocity of the ith particle, X i (t) is the current position information of the ith particle, w is an inertia weight, r represents the average uncertainty of the selected feature of the current solution, P i best (t) is the current individual optimal solution of the ith particle, G i best (t) is the global optimal solution, w*V i (t) indicates that the ith optimal solution is close to; the higher the quality of the solution, the smaller the value of r, the smaller the value of w, and the greater the probability that the ith particle performs local search around the current solution; on the contrary, the worse the quality of the solution, the greater the value of w, and the greater the probability that the ith particle performs global search.

[0122] The embodiment of the application can be based on a multi-objective particle swarm optimization feature selection algorithm, which initializes the decision variable in combination with the conditional entropy in the mode of supervised learning, and simultaneously adaptively updates the decision variable.

[0123] For example, it is assumed that there are M particles in a population, which search in a D-dimensional space. The position information of the ith particle in the tth generation is velocity The individual optimal solution is denoted as The global optimal solution of the particle swarm is denoted as G best (t) = {g1,..., g j ,..., g D}, then the updating formula of the particle velocity V i (t+1) is:

[0124]

[0125] w = 0.9-0.5exp(-t) (2)

[0126]

[0127] The value of the weight w in formula (2) can be determined based on formula (1), and then the value of the weight w is substituted into formula (3) to update the particle velocity formula V i (t+1).

[0128] By using the method provided in the embodiments of the present application, the selected probability corresponding to the features can be calculated based on the conditional entropy corresponding to the feature set, and the initialization method based on the conditional entropy is initialized based on the uncertainty information of each feature, which fully utilizes the prior information of each feature of the training sample, reduces the probability of blind search, improves the initialization quality, and the advantage of the initialization method based on the conditional entropy is particularly obvious in high-dimensional feature selection. Meanwhile, in the embodiments of the present application, the adaptive particle updating strategy updates the particle speed information and position information based on the quality of the current solution, and the quality of the current solution determines the range of particle search, and the higher the quality of the current solution, the greater the probability of local search of the particle; on the contrary, the greater the probability of global search. Therefore, the adaptive updating method further improves the quality of the solution and improves the convergence of the algorithm.

[0129] The method provided in the embodiments of the present application can be applied to any application scenario requiring image classification. Specifically, for example, in a traffic application scenario, it is necessary to distinguish vehicles and pedestrians, and the feature selection method provided in the embodiments of the present application can be applied to classify the captured image as "pedestrian" or "vehicle", and the driving personnel is reminded through the classification result to ensure traffic safety.

[0130] Based on the same inventive concept, according to the feature selection method provided in the above embodiments of the present application, correspondingly, another embodiment of the present application further provides a feature selection device, a structure diagram of which is shown in Figure 2 Specifically, it comprises:

[0131] The sample acquisition module 201 is configured to acquire a training sample set, wherein the training sample set comprises a plurality of samples, and each sample comprises D-dimensional features.

[0132] The calculation module 202 is configured to calculate the conditional entropy corresponding to each dimension of features in a feature set composed of the same dimension features of the plurality of samples, and calculate the selected probability corresponding to the features of the dimension based on the conditional entropy.

[0133] The initialization module 203 is configured to initialize a preset number of particles by a particle swarm optimization algorithm based on the selected probability corresponding to the features of each dimension, wherein each particle corresponds to a decision variable, and the decision variable comprises D elements, each element corresponding to a feature of a dimension.

[0134] The feature selection module 204 is configured to, for each element in each particle, if the value of the element is greater than a preset feature threshold, determine that the feature corresponding to the element in the particle is selected, and obtain a group of selected feature subsets corresponding to the particle.

[0135] The target function value determination module 205 is configured to determine a value of a target function based on the selected feature subset corresponding to each particle, and the value of the target function is used to represent the feature selection ratio and the feature classification error rate of each sample.

[0136] The target feature determination module 206 is configured to determine whether the target function converges based on the value of the target function, and if the target function converges, the selected feature of each sample is determined as the target feature; if the target function does not converge, each particle is updated, and the step of determining that the feature corresponding to each element in the particle is selected if the value of the element is greater than a preset feature threshold is returned.

[0137] The device provided by the embodiment of the present application is used to obtain a training sample set, calculate the conditional entropy corresponding to each dimension feature in a feature set composed of the same dimension features of a plurality of samples, and calculate the selected probability corresponding to the feature of each dimension based on the conditional entropy; a preset number of particles are initialized to optimize the decision variable based on the selected probability corresponding to each feature set, and the feature corresponding to each element in the decision variable is determined to be selected if the value of the element is greater than a preset feature threshold, so as to obtain a selected feature subset corresponding to the decision variable; the value of the target function is determined based on the feature subset corresponding to each decision variable, and if the target function converges, the selected feature of each sample is determined as the target feature; if the target function does not converge, each decision variable is updated, and the step of determining that the feature corresponding to each element in the decision variable is selected if the value of the element is greater than a preset feature threshold is returned. The method provided by the embodiment of the present application can calculate the selected probability corresponding to the feature of each dimension based on the conditional entropy corresponding to the feature of the dimension, that is, initialize the particle based on the conditional entropy, utilize the prior information of each feature of the sample, reduce the probability of blind search, improve the initialization quality, and further improve the convergence speed of the machine learning algorithm.

[0138] Optionally, the calculation module 202 is specifically configured to calculate the conditional entropy corresponding to the feature of the dimension by using the following formula:

[0139]

[0140] wherein, L is a category set to which the samples included in the training sample set belong, F m is a feature set composed of the mth dimension feature of the plurality of samples in the training sample set, H(L|F m ) is the conditional entropy corresponding to the feature Fm of the dimension, is F ma sub-event of the t-th cluster result obtained by a clustering algorithm, and l is a sub-event of L, is a joint probability, represents that a sample in a training sample set belongs to a sub-event and a class l, is a conditional probability, represents a probability that a sample belongs to a class l under a condition of a sub-event .

[0141] Optionally, the calculation module is specifically configured to calculate the selected probability corresponding to the feature of the dimension based on the conditional entropy according to the following formula:

[0142]

[0143] wherein, P(L|F m ) is the selected probability corresponding to the feature F m of the dimension, L is a set of classes to which samples in a training sample set belong, F m is a feature set composed of the m-th feature of a plurality of samples in the training sample set, H(L|F m ) is a conditional entropy corresponding to the feature of the dimension, and D is a feature dimension included by a sample.

[0144] Optionally, the initialization module 203 is specifically configured to set a preset number of particles corresponding to decision variables including D elements; for each element in each particle, a uniform random number is generated, if the uniform random number is less than the selected probability of the feature corresponding to the element, a value greater than a preset feature threshold and less than or equal to 1 is assigned to the element of the particle, otherwise, a value greater than zero and less than the preset feature threshold is assigned to the element of the particle; and a preset number of initialized particles are obtained.

[0145] Optionally, the objective function is expressed according to the following formula:

[0146]

[0147] wherein, X={x1,x2,…,x j ,…,x D} is a decision variable, is a selected feature ratio of the decision variable X, x j is the j-th element of the decision variable X, threshold is a preset feature threshold, D is a dimension of features included by a sample, and N is a number of samples in a training sample set, represents a classification error rate of classification by using a feature set selected by the decision variable X, a k represents a k-th sample in a training sample set, and a k(X) represents a sample a adopting a selected feature of decision variable X k , l k is a true class label of a k , f(a k (X)) represents a class label obtained after classification of sample a k (X), if the result of classification of sample a k (X) is same as the result of true class label, then δ(f(a k (X)), l k ) = 0, otherwise δ(f(a k (X)), l k ) = 1.

[0148] Optionally, the target feature determination module 206 is specifically configured to determine current speed information and current position information of each particle; and determine the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle, to obtain updated position information of the particle.

[0149] Optionally, the target feature determination module 206 is specifically configured to determine the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle by using the following formula:

[0150]

[0151] w = 0.9-0.5exp(-r);

[0152]

[0153] wherein H(L|F m ) is a conditional entropy corresponding to feature F m , threshold is a preset feature threshold, x m is the mth element of the current particle, t is the number of iterations, V i (t+1) is the speed of the ith particle at the t+1 generation, r1 and r2 are random numbers between 0 and 1, V i (t) is the current speed of the ith particle, X i (t) is the current position information of the ith particle, w is an inertia weight, r represents the average uncertainty of the selected feature of the current solution, P i best (t) is the current individual optimal solution of the ith particle, G i best (t) is a global optimal solution, w*V i (t) indicates the current speed of the ith The better the solution, the smaller the r value and w value; the higher the quality of the solution, the greater the probability that the i-th particle will perform a local search around the current solution. Conversely, the worse the quality of the solution, the larger the w value, and the greater the probability that the i-th particle will perform a global search. Using the apparatus provided in this embodiment, the selection probability corresponding to a certain feature can be calculated based on the conditional entropy of that feature. The initialization method based on conditional entropy initializes based on the uncertainty information of each feature, fully utilizing the prior information of each feature in the training samples, reducing the probability of blind search, and improving the initialization quality. Furthermore, the advantages of the initialization method based on conditional entropy are particularly evident in high-dimensional feature selection. In this embodiment, the adaptive particle update strategy adaptively updates the particle velocity and position information based on the quality of the current solution. The quality of the current solution determines the range of particle search; the higher the quality of the current solution, the greater the probability that the particle will perform a local search; conversely, the greater the probability that it will perform a global search. Therefore, this adaptive update method further improves the quality of the solution and enhances the convergence of the algorithm.

[0154] This invention also provides an electronic device, such as... Figure 3 As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304.

[0155] Memory 303 is used to store computer programs;

[0156] When processor 301 executes a program stored in memory 303, it performs the following steps:

[0157] Obtain a training sample set, wherein the training sample set includes multiple samples, and each sample includes D-dimensional features;

[0158] For each dimension of a feature set consisting of features of the same dimension from multiple samples, calculate the conditional entropy corresponding to that dimension of the feature, and calculate the selection probability corresponding to that dimension of the feature based on the conditional entropy.

[0159] Based on the selection probability corresponding to the features of each dimension, a preset number of particles are initialized through the particle swarm optimization algorithm. Each particle corresponds to a decision variable, which includes D elements, each element corresponding to a feature of one dimension.

[0160] For each element in each particle, if the value of the element is greater than the preset feature threshold, the feature corresponding to the element in the particle is selected, and a set of selected feature subsets corresponding to the particle is obtained.

[0161] Determine a value of the objective function based on the selected feature subset corresponding to each particle, the value of the objective function being used to represent a feature selection ratio and a feature classification error rate of each sample;

[0162] Determine whether the objective function converges based on the value of the objective function;

[0163] If the objective function converges, determine the selected features of each sample as target features;

[0164] If the objective function does not converge, update each particle, and return to the step of determining that the feature corresponding to each element in the particle is selected if the value of the element is greater than a preset feature threshold.

[0165] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0166] The communication interface is used for communication between the above electronic device and other devices.

[0167] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0168] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0169] In yet another embodiment provided by the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of any of the feature selection methods described above.

[0170] In yet another embodiment provided by the present application, a computer program product containing instructions which, when executed on a computer, cause the computer to perform any of the feature selection methods described in the above embodiments.

[0171] In the above embodiments, the implementation can be wholly or partially in software, hardware, firmware, or any combination thereof. When implemented in software, the implementation can be in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, the computer program instructions cause the computer to perform the processes or functions described in the embodiments of the present application. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, from a website site, a computer, a server, or a data center to another website site, a computer, a server, or a data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)), etc.

[0172] It should be noted that in this document, the terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0173] Various embodiments are described with reference to the related drawings. Identical or similar components that are shown in each figure are represented with the same reference numerals, and any explanation of the same components will not be repeated. In particular, for the device, electronic device, and storage medium, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the part of the method embodiment.

[0174] The above descriptions are only the preferred embodiments of the present application, not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A feature selection method characterized by, The method is applied to image classification, and the method comprises: obtaining a training sample set, wherein the training sample set comprises a plurality of samples, each sample comprises D-dimensional features, the sample is a sample image to be classified, and the D-dimensional features are D-dimensional features of the sample image to be classified; for each dimension of features in a feature set composed of the same dimension features of the plurality of samples, calculating a conditional entropy corresponding to the features of the dimension, and calculating a selected probability corresponding to the features of the dimension based on the conditional entropy; based on the selected probability corresponding to the features of each dimension, initializing a preset number of particles through a particle swarm optimization algorithm, wherein each particle corresponds to a decision variable, the decision variable comprises D elements, and each element corresponds to the features of a dimension; for each element in each particle, if a value of the element is greater than a preset feature threshold, determining that the features corresponding to the element in the particle are selected, and obtaining a selected feature subset corresponding to the particle; determining a value of a target function based on the selected feature subset corresponding to each particle, wherein the value of the target function is used to represent a feature selection ratio and a feature classification error rate of each sample; determining whether the target function converges based on the value of the target function; if the target function converges, determining selected features of each sample as target features; if the target function does not converge, determining current speed information and current position information of each particle, determining a speed of each element of the particle at a next moment based on the current speed information and the current position information of each particle, obtaining updated position information of the particle, and returning to the step of determining that the features corresponding to the element in the particle are selected if a value of the element is greater than a preset feature threshold; wherein the step of determining the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle comprises: determining the speed of each element of the particle at the next moment based on the current speed information and the current position information of each particle by using the following formula: w = 0.9 - 0.5exp(-r); Among them, H(L|F m ) is a feature F m The corresponding conditional entropy, threshold is the preset feature threshold, x m Let V be the m-th element of the current particle, t be the number of iterations, and V be the value of V. i (t+1) represents the velocity of the i-th particle in generation t+1, r1 and r2 are random numbers between 0 and 1, and V i (t) represents the current velocity of the i-th particle, X i (t) represents the current position information of the i-th particle, w is the inertial weight, r represents the average uncertainty of the feature selected in the current solution, and P i best (t) represents the current optimal solution for the i-th particle, G i best (t) represents the global optimal solution, w*V i (t) represents the i-th... The better the solution, the smaller the value of r and the smaller the value of w, and the greater the probability that the i-th particle will perform a local search around the current solution; conversely, the worse the quality of the solution, the larger the value of w, and the greater the probability that the i-th particle will perform a global search.

2. The method of claim 1, wherein, the step of calculating the conditional entropy corresponding to the features of the dimension comprises: calculating the conditional entropy corresponding to the features of the dimension by using the following formula: wherein L is a class set to which the samples included in the training sample set belong, F m is a feature set composed of the mth dimension features of the plurality of samples in the training sample set, H(L|F m is a conditional entropy corresponding to the feature Fm of the dimension, represents a sub-event of F m , F m is a tth class clustering result obtained by a clustering algorithm, and l is a sub-event of L, is a joint probability, represents a probability that a sample in the training sample set belongs to both the sub-event and the lth class, is a conditional probability, represents a probability that a sample belongs to the lth class under the condition of the sub-event .

3. The method of claim 1, wherein, the step of calculating the selected probability corresponding to the features of the dimension based on the conditional entropy comprises: calculating the selected probability corresponding to the features of the dimension based on the conditional entropy by using the following formula: wherein P(L|F m ) is the probability of the feature F m in the dimension corresponding to the selected probability, L is the class set to which the sample included in the training sample set belongs, F m is a feature set composed of the mthdimensional features of the plurality of samples in the training sample set, H(L|F m ) is the conditional entropy corresponding to the feature in the dimension, and D is the feature dimension included in the sample.

4. The method of claim 1, wherein, the step of initializing the preset number of particles based on the selected probability corresponding to the features of each dimension through the particle swarm optimization algorithm comprises: setting particles corresponding to a preset number of decision variables comprising D elements; for each element in each particle, generating a uniform random number, if the uniform random number is less than the selected probability of the features corresponding to the element, assigning a value greater than the preset feature threshold and less than or equal to 1 to the element of the particle, otherwise, assigning a value greater than zero and less than the preset feature threshold to the element of the particle; and obtaining the preset number of initialized particles.

5. The method of claim 1, wherein, the target function is represented by using the following formula: where X = {x1, x2, …, x j} is the decision variable, D} is the decision variable, is the selected feature ratio of the decision variable X, x j is the jth element of the decision variable X, threshold is a preset feature threshold, D is the dimension of the features included in the sample, and N is the number of samples in the training sample set, represents the classification error rate using the selected feature set of the decision variable X, a k represents the kth sample in the training sample set, a k (X) represents the sample a k using the selected features of the decision variable X, l k is the true class label of a k , and f(a k (X)) represents the class label obtained after the sample a k (X) is classified. If the classification result of the sample a k (X) is the same as the true class label result, then δ(f(a k (X)), l k ) = 0, otherwise δ(f(a k (X)), l k ) = 1.

6. A feature selection apparatus characterized by comprising: the device is applied to image classification, and the device comprises: a sample obtaining module, configured to obtain a training sample set, wherein the training sample set comprises a plurality of samples, each of the samples comprises D-dimensional features, and each of the samples is a sample image to be classified, and the D-dimensional features are D-dimensional features of the sample image to be classified; a calculating module, configured to calculate, for each dimension of features in a feature set composed of the same dimension features of the plurality of samples, a conditional entropy corresponding to the features of the dimension, and calculate a selected probability corresponding to the features of the dimension based on the conditional entropy; an initializing module, configured to initialize a preset number of particles by a particle swarm optimization algorithm based on the selected probabilities corresponding to the features of the dimensions, wherein each of the particles corresponds to a decision variable, the decision variable comprises D elements, and each of the elements corresponds to features of a dimension; a feature selecting module, configured to, for each element in each of the particles, if a value of the element is greater than a preset feature threshold, determine that features corresponding to the element in the particle are selected, and obtain a selected feature subset corresponding to the particle; a target function value determining module, configured to determine a value of a target function based on the selected feature subsets corresponding to the particles, wherein the value of the target function is used to represent a feature selection ratio of each of the samples and a feature classification error rate; a target feature determining module, configured to determine whether the target function converges based on the value of the target function, if the target function converges, determine selected features of each of the samples as target features, and if the target function does not converge, determine current speed information and current position information of each of the particles, determine a speed of each element of the particle at a next moment based on the current speed information and the current position information of each of the particles, obtain updated position information of the particle, and return the step of, for each element in each of the particles, if a value of the element is greater than a preset feature threshold, determining that features corresponding to the element in the particle are selected; the target feature determining module is specifically configured to determine the speed of each element of the particle at the next moment based on the current speed information and the current position information of each of the particles by using the following formula: w = 0.9 - 0.5exp(-r); where H(L|F m ) is the conditional entropy of feature F m , threshold is a pre-set feature threshold, x m is the mth element of the current particle, t is the iteration number, V i (t+1) is the velocity of the ith particle in the t+1 generation, r1 and r2 are random numbers between 0 and 1, V i (t) is the current velocity of the ith particle, X i (t) is the current position information of the ith particle, w is the inertia weight, r represents the average uncertainty of the selected feature of the current solution, P i best (t) is the current individual optimal solution of the ith particle, G i best (t) is the global optimal solution, w*V i (t) indicates that the ith particle is close to the current solution; the higher the quality of the solution, the smaller the value of r and w, and the greater the probability of the ith particle performing local search around the current solution; on the contrary, the worse the quality of the solution, the greater the value of w, and the greater the probability of the ith particle performing global search.

7. An electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; the memory is used to store a computer program; the processor is used to execute the program stored on the memory, and implement the method steps in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 1-5.

Images