Feature Selection Method, Apparatus and Device

Abstract

The present invention provides a feature selection method, apparatus and device. The method includes: obtaining an initial position vector of each Harris hawk based on the total number of multiple candidate features in a target dataset; determining a target position vector based on a feature indication function model, a Harris hawk fitness function model and the initial position vector of each Harris hawk; converting the target position vector into a target indication vector through the feature indication function model, where the target indication vector includes indication values corresponding to the multiple candidate features respectively, and when the indication value is a preset value, it is used to represent that the corresponding candidate feature is selected; and selecting the candidate feature corresponding to the preset value among the multiple candidate features as the target feature. The feature selection method, apparatus and device provided by the embodiments of the present invention are used to solve the problem of selecting valuable features from multiple features of a dataset.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine learning technology and data processing technology, and particularly relates to a feature selection method, device, and equipment. Background Art

[0002] Currently, an initial model can be trained with a data set to obtain an application model that can be used for actual prediction.

[0003] In the related art, in order to improve the prediction accuracy of the application model, a data set with a larger scale (referring to more rows in the data set, that is, the data set includes multiple pieces of data) and higher complexity (referring to more columns in the data set, that is, each piece of data in the data set has more features) is used to train the initial model.

[0004] In the above-mentioned related art, using a data set with a larger scale and higher complexity to train the initial model will cause problems such as overfitting, degradation of model performance, and increased computational complexity.

[0005] Therefore, how to select valuable features from multiple features of the data set to avoid problems such as overfitting, degradation of model performance, and increased computational complexity during the model training process has become a technical problem to be urgently solved. Summary of the Invention

[0006] The present invention provides a feature selection method, device, and equipment to solve the problem of selecting valuable features from multiple features of a data set.

[0007] In a first aspect, the present invention provides a feature selection method, including:

[0008] Based on the total number of multiple candidate features in the target data set, obtain the initial position vectors of each Harris hawk, where the initial position vectors include the total number of positions;

[0009] Based on the feature indicator function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk, determine the target position vector;

[0010] Through the feature indicator function model, convert the target position vector into a target indicator vector, where the target indicator vector includes the indicator values corresponding to each of the multiple candidate features, and when the indicator value is a preset value, it is used to represent that the corresponding candidate feature is selected;

[0011] Select the candidate features corresponding to the preset value among the multiple candidate features as the target features.

[0012] According to one provided by the present invention, determining the target position vector based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of the respective Harris hawks includes:

[0013] Perform the following operations for the i-th iteration: Obtain the position vectors i of the respective Harris hawks; through the feature indication function model, convert the position vectors i of the respective Harris hawks into the indication vectors i of the respective Harris hawks; determine the accuracy of the indication vectors i of the respective Harris hawks; through the Harris hawk fitness function model, process the accuracy of the indication vectors i of the respective Harris hawks to obtain the fitness corresponding to the position vectors i of the respective Harris hawks; determine the position vector with the minimum fitness among the respective position vectors i and the candidate position vector (i - 1) as the candidate position vector i; update the position vectors i of the respective Harris hawks to obtain the position vectors (i + 1) of the respective Harris hawks;

[0014] Increment i by 1, and repeat the above operations until, when i is equal to the preset number of iterations, determine the candidate position vector i as the target position vector; initially, i is equal to 1, and the position vector i of the Harris hawk is the initial position vector of the Harris hawk.

[0015] According to one feature selection method provided by the present invention, the Harris hawk fitness function model is:

[0016]

[0017] Where M Q represents the position vector i, F(M Q ) is the fitness corresponding to the position vector i, α is a preset adjustment parameter value, is the accuracy determined based on the indication vector i, |M Q | represents the total number of the preset values in the indication vector i, and |M| represents the total number of the multiple candidate features.

[0018] According to one feature selection method provided by the present invention, updating the position vector i of the Harris hawk to obtain the position vector (i + 1) of the Harris hawk includes:

[0019] Obtain the escape probability;

[0020] Based on the escape probability, i, and the preset number of iterations, determine the escape energy;

[0021] Based on the escape probability and the escape energy, determine the update strategy;

[0022] Based on the update strategy, update the position vector i of the Harris hawk to obtain the position vector (i + 1) of the Harris hawk.

[0023] According to a feature selection method provided by the present invention, determining the escape energy based on the escape probability, i, and the preset number of iterations includes:

[0024] By processing the escape probability, i, and the preset number of iterations to obtain the escape energy;

[0025] Wherein, E represents the escape energy, f represents the escape probability, and I represents the preset number of iterations.

[0026] According to a feature selection method provided by the present invention, the feature indication function model is:

[0027]

[0028] Wherein, X Q represents the target position vector, x q represents the q-th position in X Q , x q ∈X Q , T(x q ) represents the probability that the candidate feature corresponding to x q is selected, M Q represents the target indication vector, rand(0,1) represents a random number generation function, and the random number generation function is used to generate a random number between 0 and 1.

[0029] In a second aspect, the present invention further provides a feature selection device, including:

[0030] An acquisition module, configured to acquire an initial position vector of each Harris hawk based on the total number of multiple candidate features in a target dataset, where the initial position vector includes the total number of positions;

[0031] A determination module, configured to determine a target position vector based on a feature indication function model, a Harris hawk fitness function model, and the initial position vectors of the Harris hawks;

[0032] A conversion module, configured to convert the target position vector into a target indication vector through the feature indication function model, where the target indication vector includes indication values corresponding to the multiple candidate features respectively, and when the indication value is a preset value, it is used to represent that the corresponding candidate feature is selected;

[0033] A selection module, configured to select the candidate feature corresponding to the preset value among the multiple candidate features as the target feature.

[0034] According to a feature selection device provided by the present invention, the determination module is specifically configured to:

[0035] For the i-th iteration, perform the following operations: Obtain the position vector i of each Harris hawk; through the feature indication function model, convert the position vector i of each Harris hawk into the indication vector i of each Harris hawk; determine the accuracy rate of the indication vector i of each Harris hawk; through the Harris hawk fitness function model, process the accuracy rate of the indication vector i of each Harris hawk to obtain the fitness corresponding to the position vector i of each Harris hawk; determine the position vector with the minimum fitness among each position vector i and the candidate position vector (i - 1) as the candidate position vector i; update the position vector i of each Harris hawk to obtain the position vector (i + 1) of each Harris hawk;

[0036] Increment i by 1, and repeat the above operations until when i is equal to the preset number of iterations, determine the candidate position vector i as the target position vector; initially, i is equal to 1, and the position vector i of the Harris hawk is the initial position vector of the Harris hawk.

[0037] According to a feature selection device provided by the present invention, the Harris hawk fitness function model is:

[0038]

[0039] where M Q represents the position vector i, F(M Q ) is the fitness corresponding to the position vector i, α is a preset adjustment parameter value, is the accuracy rate determined based on the indication vector i, |M Q | represents the total number of the preset values in the indication vector i, and |M| represents the total number of the multiple candidate features.

[0040] According to a feature selection device provided by the present invention, the determining module is specifically configured to:

[0041] Obtain the escape probability;

[0042] Based on the escape probability, i, and the preset number of iterations, determine the escape energy;

[0043] Based on the escape probability and the escape energy, determine the update strategy;

[0044] Based on the update strategy, update the position vector i of the Harris hawk to obtain the position vector (i + 1) of the Harris hawk.

[0045] According to a feature selection device provided by the present invention, the determining module is specifically configured to:

[0046] Through Process the escape probability, i, and the preset number of iterations to obtain the escape energy;

[0047] Where E represents the escape energy, f represents the escape probability, and I represents the preset number of iterations.

[0048] According to a feature selection device provided by the present invention, the feature indication function model is:

[0049]

[0050] Where X Q represents the target position vector, x q represents the q-th position in X Q , x q ∈X Q , T(x q ) represents the probability that the candidate feature corresponding to x q is selected, M Q represents the target indication vector, rand(0, 1) represents a random number generation function, and the random number generation function is used to generate a random number between 0 and 1.

[0051] In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the feature selection method as described in any one of the above.

[0052] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the feature selection method as described in any one of the above.

[0053] In a fifth aspect, the present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the feature selection method as described in any one of the above.

[0054] A feature selection method, device, and equipment provided by the present invention, by obtaining the initial position vectors of each Harris hawk based on the total number of multiple candidate features in the target dataset, where the initial position vectors include the total number of positions; determining the target position vectors based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk; converting the target position vectors into target indication vectors through the feature indication function model, where the target indication vectors include the indication values corresponding to the multiple candidate features respectively, and when the indication value is a preset value, it is used to represent that the corresponding candidate feature is selected; selecting the candidate feature corresponding to the preset value among the multiple candidate features as the target feature, solves the problem of selecting valuable features from multiple features in the dataset. Description of the Drawings

[0055] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0056] Figure 1 is a schematic flowchart of the feature selection method provided by the embodiment of the present invention;

[0057] Figure 2 is a schematic diagram of the image of T(x q ) provided by the embodiment of the present invention;

[0058] Figure 3 is a schematic flowchart of the method for determining the target position vector provided by the embodiment of the present invention;

[0059] Figure 4 is a schematic flowchart of the method for obtaining the position vector (i + 1) of the Harris hawk provided by the embodiment of the present invention;

[0060] Figure 5 is a schematic diagram of the feature selection device provided by the embodiment of the present invention;

[0061] Figure 6 is a schematic diagram of the physical structure of the electronic device provided by the embodiment of the present invention. Detailed implementation manners

[0062] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0063] In the present invention, the term "including" and its variants may refer to non-restrictive inclusion; the term "or" and its variants may refer to "and / or". In the present invention, terms such as "first" and "second" are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. In the present invention, "at least one" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0064] In the related art, when training an initial model using a dataset with a large scale and high complexity, problems such as overfitting, degradation of model performance, and increased computational complexity may occur. Therefore, how to select valuable features from multiple features of the dataset to avoid problems such as overfitting, degradation of model performance, and increased computational complexity during model training has become a technical problem to be solved urgently.

[0065] To solve the above technical problems, the present invention provides a feature selection method. The application scenarios of the feature selection method provided by the present invention can be intrusion detection, image classification, etc. The following will be combined with Figures 1 to 4 embodiments to describe the feature selection method provided by the present invention.

[0066] Figure 1 is a schematic flowchart of the feature selection method provided by an embodiment of the present invention. As Figure 1 shown, the method includes:

[0067] Step 101: Based on the total number of various candidate features in the target dataset, obtain the initial position vectors of each Harris hawk, and the initial position vectors include the total number of positions.

[0068] Optionally, for the feature selection method provided by the present invention, the execution subject can be an electronic device or a feature selection device provided in the electronic device. The feature selection device can be implemented by a combination of software and / or hardware. For example, the electronic device can be a desktop computer, a laptop computer, a tablet computer, a mobile phone, etc.

[0069] In intrusion detection, the target dataset is, for example, NSL-KDD, KDD99, or UNSW-NB15, etc.

[0070] In image classification, the target dataset is, for example, CIFAR10, ImageNet, or WIDER Face, etc.

[0071] Taking intrusion detection as an example below, the target dataset will be exemplarily described through Table 1.

[0072] Table 1

[0073] B1 B2 …… By D1 C11 C12 …… C1y D2 C21 C22 …… C2y …… …… …… …… …… Dz Cz1 Cz2 …… Czy

[0074] Table 1 includes z rows and y columns. Among them, the z rows indicate that the target dataset includes z pieces of data. For example, the z pieces of data include D1... Dz, and the y columns indicate that the target dataset includes y kinds of candidate features. For example, the y kinds of candidate features include B1... By. Each kind of candidate feature includes z feature values. For example, the candidate feature B1 includes feature values C11... Cz1.

[0075] Specifically, when the target dataset is UNSW-NB15, B1 is srcip, B2 is sport, C11 is 59.166.0.9, and C12 is 7045.

[0076] Optionally, the number of Harris hawks can be multiple.

[0077] Optionally, the number of Harris hawks can be obtained through customization or randomly generated.

[0078] Optionally, for each Harris hawk, according to the total number of multiple candidate features, perform random initialization to obtain the initial position vector of the Harris hawk, so that the initial position vector includes the total number of positions.

[0079] For example, if the total number of multiple candidate features is y, the initial position vector includes y positions.

[0080] Step 102: Determine the target position vector based on the feature indicator function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk.

[0081] Step 103: Through the feature indicator function model, convert the target position vector into a target indicator vector. The target indicator vector includes the indicator values corresponding to each of the multiple candidate features. When the indicator value is a preset value, it is used to represent the selection of the corresponding candidate feature.

[0082] Optionally, the indicator value can be 0 or 1. For example, when the indicator value is 1, it can represent the selection of the corresponding candidate feature; or when the indicator value is 0, it represents the selection of the corresponding candidate feature.

[0083] Step 104: Select the candidate features corresponding to the preset value among the multiple candidate features as the target features.

[0084] For example, if the multiple candidate features include B1, B2, B3, and B4, and the preset value is 1, then when the target indicator vector is [0, 1, 1, 0], B2 and B3 are determined as the target features.

[0085] Furthermore, for each piece of data in the target dataset, obtain the feature values corresponding to the target features from this piece of data to obtain new data; and train the initial model based on the multiple pieces of new data to obtain an application model.

[0086] For example, for the data D1 in the target dataset shown in Table 1, if B1 and B2 are the target features, then C11 corresponding to B1 and C12 corresponding to B2 in D1 are determined as the new data.

[0087] For example, in intrusion detection, the initial model is a multi-layer perceptron.

[0088] InFigure 1 In the embodiment, based on the total number of various candidate features in the target dataset, the initial position vectors of each Harris hawk are obtained. The initial position vectors include the total number of positions. Based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk, the target position vector is determined. Through the feature indication function model, the target position vector is converted into a target indication vector. The target indication vector includes the indication values corresponding to various candidate features respectively. The indication value is a preset value used to represent the candidate feature corresponding to the selected position. The candidate feature corresponding to the preset value among the various candidate features is selected as the target feature, which can select valuable features from multiple features in the dataset, eliminate redundant and irrelevant features, so as to avoid problems such as overfitting, model performance degradation, and increased computational complexity during model training.

[0089] In some embodiments, the feature indication function model is:

[0090]

[0091] where X Q represents the target position vector, x q represents the q-th position in X Q , x q ∈X Q , T(x q ) represents the probability that the candidate feature corresponding to x q is selected, M Q represents the target indication vector, and rand(0,1) represents a random number generation function, which is used to generate a random number between 0 and 1.

[0092] Next, in combination with Figure 2 , the feature indication function model will be described.

[0093] Figure 2 is the schematic diagram of the image of T(x q ) provided by the embodiment of the present invention. As Figure 2 shown, T(x q ) is an S-shaped function (sigmoid function). T(x q ) is a conversion function that can convert the position of the Harris hawk (continuous value) into the probability that the candidate feature corresponding to the position is selected.

[0094] It can be seen from the image that when x q is -0.58, the probability that the q-th position in the target position vector is selected is 23.87%; when x q is 1.57, the probability that the q-th position in the target position vector is selected is 95.85%.

[0095] M Q is an update function. When the position of the Harris hawk is converted to the probability that the candidate feature corresponding to the position is selected, based on M Q update the probability that the candidate feature is selected to an indication value (discrete value).

[0096] For example, when rand(0, 1) is equal to 0.5, when T(x q ) is greater than 0.5, determine the indication value of position x q as 1; when T(x q ) is less than or equal to 0.5, determine the indication value of position x q as 0.

[0097] In the embodiments of the present invention, through the feature indication function model, the target position vector is converted into a target indication vector. The target indication vector includes indication values corresponding to various candidate features respectively. The indication value is a preset value used to characterize the candidate feature corresponding to the selected position, which can solve the mapping problem between the position of the Harris hawk and the selection situation of the candidate feature corresponding to the position, so that the search space of the Harris Hawk Optimization (HHO) algorithm is various candidate features, so that the HHO algorithm can be applied to feature selection, and then valuable features can be selected from multiple features of the data set to avoid problems such as overfitting, model performance degradation and increased computational complexity.

[0098] Next, in combination with Figure 3 embodiments, a method for determining the target position vector will be described.

[0099] Figure 3 is a schematic flowchart of the method for determining the target position vector provided by the embodiments of the present invention. As Figure 3 shown, the method includes:[[]]

[0100] Step 301, obtain the position vector i of each Harris hawk.

[0101] Initially, the position vector i of the Harris hawk is the initial position vector of the Harris hawk.

[0102] Step 302, through the feature indication function model, convert the position vector i of each Harris hawk into the indication vector i of each Harris hawk; determine the accuracy rate of the indication vector i of each Harris hawk; through the Harris hawk fitness function model, process the accuracy rate of the indication vector i of each Harris hawk to obtain the fitness corresponding to the position vector i of each Harris hawk; determine the position vector with the minimum fitness among the position vector i and the candidate position vector (i - 1) as the candidate position vector i.

[0103] The feature indication function model is asFigure 2 As shown in the embodiments, the target position vector is converted into a target indication vector through the feature indication function model. Details are not described herein again.

[0104] In some embodiments, the Harris hawk fitness function model is:

[0105]

[0106] Where M Q represents the position vector i, F(M Q ) is the fitness corresponding to the position vector i, α is a preset adjustment parameter value, is the accuracy rate determined based on the indication vector i, |M Q | represents the total number of preset values in the indication vector i, and |M| represents the total number of multiple candidate features.

[0107] In some embodiments, determining the accuracy rate of the indication vector i of the Harris hawk includes:

[0108] For each Harris hawk, based on the indication vector i of the Harris hawk, determine the features corresponding to the indication vector i;

[0109] Based on the feature values of the features corresponding to the indication vector i, train a preset model to obtain a trained preset model;

[0110] Input the feature values of the features corresponding to the indication vector i into the trained preset model to obtain the accuracy rate of the indication vector i.

[0111] The preset model is, for example, a multi-layer perceptron.

[0112] For example, if the indication vector i is [1, 1, 0, 0], and the multiple candidate features include B1, B2, B3, and B4, then B1 and B2 are the features corresponding to the indication vector i; based on the feature values C11 to Cz1 of B1 and the feature values C12 to Cz2 of B2, train the multi-layer perceptron to obtain a trained multi-layer perceptron; input C11 to Cz1 and C12 to Cz2 into the trained multi-layer perceptron to obtain the accuracy rate of the indication vector i.

[0113] In the embodiments of the present invention, the Harris hawk fitness function model includes the accuracy rate and the quantity (|M Q | and |M|) as two influencing factors, defines the evaluation criteria for candidate position vectors, and by adjusting the value of α, it is possible to determine a candidate position vector that is more biased towards one of the two influencing factors.

[0114] Step 303: Update the position vector i of each Harris hawk to obtain the position vector (i + 1) of each Harris hawk.

[0115] Step 304: Determine whether i is equal to the preset number of iterations.

[0116] If so, execute Step 306; otherwise, execute Step 305.

[0117] Step 305: Increment i by 1, and repeat Steps 301 to 304.

[0118] Step 306: Determine the candidate position vector i as the target position vector.

[0119] Taking a Harris hawk as an example, in combination with Figure 4 the embodiments, Step 303 will be described in detail.

[0120] Figure 4 This is a schematic flowchart of the method for obtaining the position vector (i + 1) of a Harris hawk provided by the embodiments of the present invention. As Figure 4 shown, the method includes:

[0121] Step 401: Obtain the escape probability.

[0122] Optionally, the escape probability is a randomly generated random number with a value between 0 and 1. During each iteration, the escape probability is randomly generated.

[0123] Step 402: Determine the escape energy based on the escape probability, i, and the preset number of iterations.

[0124] In some embodiments, determining the escape energy based on the escape probability, i, and the preset number of iterations includes:

[0125] By processing the escape probability, i, and the preset number of iterations, the escape energy is obtained; where E represents the escape energy, f represents the escape probability, and I represents the preset number of iterations.

[0126] Optionally, the escape energy can also be determined by the following formula 1:

[0127]

[0128] where E0 is the initial escape energy.

[0129] Optionally, the initial escape energy is determined by the following formula 2:

[0130] E0 = 2f - 1 Formula 2;

[0131] where E0 represents the initial escape energy.

[0132] Step 403: Based on the escape probability and escape energy, determine the update strategy, and based on the update strategy, update the position vector i of the Harris hawk to obtain the position vector (i + 1) of the Harris hawk.

[0133] Optionally, the update strategy includes: the position update strategy in the exploration stage, the position update strategy in the soft encirclement stage, the position update strategy in the hard encirclement stage, the position update strategy in the soft encirclement stage of the progressive rapid dive, and the position update strategy in the hard encirclement stage of the progressive rapid dive.

[0134] For example, when |E| ≥ 1, determine that the update strategy is the position update strategy in the exploration stage.

[0135] For example, when |E| ≥ 0.5 and f ≥ 0.5, determine that the update strategy is the position update strategy in the soft encirclement stage.

[0136] For example, when |E| < 0.5 and f ≥ 0.5, determine that the update strategy is the position update strategy in the hard encirclement stage.

[0137] For example, when |E| ≥ 0.5 and f < 0.5, determine that the update strategy is the position update strategy in the soft encirclement stage of the progressive rapid dive.

[0138] For example, when |E| < 0.5 and f < 0.5, determine that the update strategy is the position update strategy in the hard encirclement stage of the progressive rapid dive.

[0139] Optionally, the position update strategy in the exploration stage is:

[0140]

[0141] where represents the position vector of the Harris hawk at the (i + 1)-th iteration, represents the position vector of another randomly selected Harris hawk at the i-th iteration, represents the position vector of the Harris hawk at the i-th iteration, |·| represents the absolute value symbol, represents the difference between the optimal position vector at the i-th iteration and the average position vector at the i-th iteration, l1, l2, l3, and l4 are random numbers generated between 0 and 1, and ΔX gap represents the difference between the upper bound of the feature selection feasible region (e.g., 1) and the lower bound of the feature selection feasible region (e.g., 0).

[0142] Optionally, can be determined by the following formula 3

[0143]

[0144] where X*,i represents the optimal position vector at the i-th iteration, represents the average position vector at the i-th iteration. The optimal position vector is the position vector with the minimum fitness among all candidate position vectors obtained in the previous i iterations, and the average position vector is the average vector of all candidate position vectors obtained in the previous i iterations.

[0145] Optionally, ΔX can be determined by the following formula 4 gap :

[0146] ΔX gap = X U - X L Formula 4;

[0147] where, X U represents the upper bound of the feature selection feasible region, and X L represents the lower bound of the feature selection feasible region.

[0148] Optionally, the position update strategy in the soft enclosing stage is:

[0149] where, represents the difference between the optimal position vector at the i-th iteration and the position vector of the Harris hawk at the i-th iteration, J is the jump intensity, J = 2(1 - f5), and f5 is a random number between 0 and 1.

[0150] Optionally, can be determined by the following formula 5

[0151]

[0152] Optionally, the position update strategy in the hard enclosing stage is:

[0153] Optionally, the position update strategy in the soft enclosing stage of the progressive fast dive is:

[0154]

[0155] where, Y and Z represent two newly generated position vectors, ε represents a random vector with dimension y, y represents the total dimension of the features, Levy represents the flight function value, and F(·) represents the Harris hawk fitness function model.

[0156] Optionally, Levy can be determined by the following formula 6:

[0157]

[0158] where, μ and v represent two independent random numbers subject to the normal distribution, σ represents the first preset value, and β represents the second preset value.

[0159] Optionally, β can be 1.5.

[0160] Optionally, σ can be determined by the following formula (7):

[0161]

[0162] where Γ(·) represents the Gamma Function.

[0163] Optionally, the position update strategy for the hard surround stage of the progressive rapid dive is as follows:

[0164]

[0165] In Figure 4 the embodiment, based on the escape probability and the escape energy, the update strategy is determined, and based on the update strategy, the position vector i of the Harris hawk is updated to obtain the position vector (i + 1) of the Harris hawk, which improves the accuracy of obtaining the position vector of the Harris hawk. Compared with the feature selection method in the prior art, the time consumed for feature selection is reduced.

[0166] Next, the embodiment of the present invention provides pseudocode to implement the above feature selection method. The detailed pseudocode is as follows:

[0167]

[0168]

[0169] Through the feature indicator function model, the target position vector X * → the target indicator vector M * ;

[0170] return M * .

[0171] In the related art, the following methods (1) to (5) are usually adopted for feature selection.

[0172] (1) Forward Selection. Use certain algorithms or models of machine learning for training to obtain the weight coefficients of each feature, and select features from large to small according to the coefficients until the performance of the model does not improve significantly.

[0173] (2) Backward Elimination, contrary to the forward feature selection, starting from the complete feature set, keeping the metric of the model unchanged, and iteratively reducing features one by one.

[0174] (3) Filter-based methods score each feature according to divergence or correlation, and select features by setting a threshold or the number of thresholds to be selected.

[0175] (4) Wrapper-based methods use machine learning algorithms to evaluate the effectiveness of feature subsets, can detect the interaction relationships between two or more features, and the selected feature subsets optimize the performance of the model.

[0176] (5) Embedded methods embed feature selection into the model construction process, and select optimal features by optimizing the objective function of the model.

[0177] As can be seen from the above methods (1) to (5), the following problems 1 to 4 exist in the related technologies:

[0178] Problem 1: (1) and (2) have limitations in identifying the interactions of complex features and processing large datasets.

[0179] Problem 2: (3) does not consider selecting feature subsets for the subsequent model to be used, weakening the fitting ability of the model.

[0180] Problem 3: (4) requires training a model for each group of feature subsets, with a large amount of computation, being prone to overfitting when the samples are insufficient, and having too high computational complexity when there are many feature variables.

[0181] Problem 4: (5) requires selecting appropriate regularization parameters, otherwise it will result in too many or too few selected features, and has poor feature selection effects for non-linear relationship features, and may select features that are highly correlated with the target variable but poorly correlated with other features, resulting in reduced interpretability of the model.

[0182] To solve the above technical problem 1, feature selection is performed through the HHO algorithm, which can identify the interactions of complex features, can handle feature selection in various different large datasets (with many features), and achieve the optimal accuracy performance under the constraint of the number of selected features, thus solving the above problem 1.

[0183] To solve the above technical problem 2, by designing a fitness function, the conflicting requirements between the reduction of the feature set and the excellent performance of the algorithm are satisfied, without considering selecting feature subsets for the subsequent model to be used, and the fitting ability of the model is improved.

[0184] To solve the above technical problem 3, by defining a transformation function The mapping relationship between the position and feature selection of Harris hawks is solved, enabling the Harris hawk algorithm to be applied to the feature selection task. There is no need to train a model for each set of feature subsets, greatly reducing the computational cost and avoiding the problems of overfitting of the model when the samples are insufficient and high computational complexity when there are many feature variables, thus solving the above problem 3.

[0185] To solve the above technical problem 4, in the feature selection method provided by the embodiments of the present invention, through the fast convergence performance of the HHO algorithm, the finally selected feature subset can meet the customized requirements (referring to meeting the conflicting requirements of the algorithm between feature set reduction and excellent performance), without the need to select appropriate regularization parameters, improving the interpretability of the model and being able to solve problem 4.

[0186] In this application, a feature subset includes a set composed of the selected features.

[0187] The following describes the feature selection device provided by the present invention. The feature selection device described below can be mutually referred to with the feature selection method described above.

[0188] Figure 5 It is a schematic diagram of the feature selection device provided by the embodiments of the present invention. As Figure 5 shown, the device includes:

[0189] An acquisition module 510, configured to obtain an initial position vector of each Harris hawk based on the total number of multiple candidate features in the target dataset, where the initial position vector includes the total number of positions;

[0190] A determination module 520, configured to determine a target position vector based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk;

[0191] A conversion module 530, configured to convert the target position vector into a target indication vector through the feature indication function model, where the target indication vector includes indication values corresponding to each of the multiple candidate features, and when the indication value is a preset value, it is used to represent the selected corresponding candidate feature;

[0192] A selection module 540, configured to select the candidate features corresponding to the preset value among the multiple candidate features as the target features.

[0193] It should be noted here that the above device provided by the embodiments of the present invention can implement all the method steps implemented by the above method embodiments and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0194] According to a feature selection device provided by the present invention, the determination module 520 is specifically configured to:

[0195] Perform the following operations for the i-th iteration: Obtain the position vector i of each Harris hawk; convert the position vector i of each Harris hawk into the indication vector i of each Harris hawk through the feature indication function model; determine the accuracy rate of the indication vector i of each Harris hawk; process the accuracy rate of the indication vector i of each Harris hawk through the Harris hawk fitness function model to obtain the fitness corresponding to the position vector i of each Harris hawk; determine the position vector with the minimum fitness among each position vector i and the candidate position vector (i - 1) as the candidate position vector i; update the position vector i of each Harris hawk to obtain the position vector (i + 1) of each Harris hawk;

[0196] Increment i by 1 and repeat the operations until, when i is equal to the preset number of iterations, determine the candidate position vector i as the target position vector; initially, i is equal to 1, and the position vector i of the Harris hawk is the initial position vector of the Harris hawk.

[0197] According to a feature selection device provided by the present invention, the Harris hawk fitness function model is:

[0198]

[0199] where M Q represents the position vector i, F(M Q ) is the fitness corresponding to the position vector i, α is the preset adjustment parameter value, is the accuracy rate determined based on the indication vector i, |M Q | represents the total number of preset values in the indication vector i, and |M| represents the total number of multiple candidate features.

[0200] According to a feature selection device provided by the present invention, the determination module 520 is specifically configured to:

[0201] Obtain the escape probability;

[0202] Determine the escape energy based on the escape probability, i, and the preset number of iterations;

[0203] Determine the update strategy based on the escape probability and the escape energy;

[0204] Update the position vector i of the Harris hawk based on the update strategy to obtain the position vector (i + 1) of the Harris hawk.

[0205] According to a feature selection device provided by the present invention, the determination module 520 is specifically configured to:

[0206] Through process the escape probability, i, and the preset number of iterations to obtain the escape energy;

[0207] Among them, E represents the escape energy, f represents the escape probability, and I represents the preset number of iterations.

[0208] According to a feature selection device provided by the present invention, the feature indication function model is:

[0209]

[0210] Among them, X Q represents the target position vector, and x q represents the q-th position in X Q , x q ∈X Q , T(x q ) represents the probability that the candidate feature corresponding to x q is selected, M Q represents the target indication vector, rand(0,1) represents the random number generation function, and the random number generation function is used to generate a random number between 0 and 1.

[0211] Figure 6 is a schematic physical structure diagram of an electronic device provided by an embodiment of the present invention. As Figure 6 shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640. Among them, the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logical instructions in the memory 630 to execute the feature selection method, and the method includes: based on the total number of various candidate features in the target dataset, obtaining the initial position vector of each Harris hawk, and the initial position vector includes the total number of positions; based on the feature indication function model, the Harris hawk fitness function model, and the initial position vector of each Harris hawk, determining the target position vector; through the feature indication function model, converting the target position vector into a target indication vector, and the target indication vector includes the indication values corresponding to various candidate features respectively, and when the indication value is a preset value, it is used to represent that the corresponding candidate feature is selected; selecting the candidate feature corresponding to the preset value among the various candidate features as the target feature.

[0212] In addition, when the logical instructions in the above-mentioned memory 630 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0213] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the feature selection method provided by the above-mentioned various methods. The method includes: based on the total number of various candidate features in the target dataset, obtaining the initial position vectors of each Harris hawk, where the initial position vectors include the total number of positions; based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk, determining the target position vector; through the feature indication function model, converting the target position vector into a target indication vector, where the target indication vector includes the indication values corresponding to each of the various candidate features, and when the indication value is a preset value, it is used to represent the selection of the corresponding candidate feature; selecting the candidate feature corresponding to the preset value among the various candidate features as the target feature.

[0214] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the feature selection method provided by the above-mentioned various methods. The method includes: based on the total number of various candidate features in the target dataset, obtaining the initial position vectors of each Harris hawk, where the initial position vectors include the total number of positions; based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of each Harris hawk, determining the target position vector; through the feature indication function model, converting the target position vector into a target indication vector, where the target indication vector includes the indication values corresponding to each of the various candidate features, and when the indication value is a preset value, it is used to represent the selection of the corresponding candidate feature; selecting the candidate feature corresponding to the preset value among the various candidate features as the target feature.

[0215] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0216] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0217] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature selection method, characterized in that, Including: Based on the total number of multiple candidate features in the target dataset, obtain the initial position vectors of each Harris hawk, where the initial position vectors include the positions corresponding to the total number; wherein, the target dataset includes CIFAR10; Determine the target position vector based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of the Harris hawks; the determining the target position vector based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of the Harris hawks includes: performing the following operations for the i-th iteration: obtaining the position vectors i of the Harris hawks; converting the position vectors i of the Harris hawks into indication vectors i of the Harris hawks through the feature indication function model; determining the accuracy rates of the indication vectors i of the Harris hawks; processing the accuracy rates of the indication vectors i of the Harris hawks through the Harris hawk fitness function model to obtain the fitness values corresponding to the position vectors i of the Harris hawks; determining the position vector with the minimum fitness value among the position vectors i and the candidate position vector (i - 1) as the candidate position vector i; updating the position vectors i of the Harris hawks to obtain the position vectors (i + 1) of the Harris hawks; adding 1 to i, and repeating the operations until, when i is equal to the preset number of iterations, determining the candidate position vector i as the target position vector; initially, i is equal to 1, and the position vector i of the Harris hawk is the initial position vector of the Harris hawk; the feature indication function model is: ; where represents the target position vector, represents the -th position in , represents the probability that the corresponding candidate feature is selected, represents the target indication vector, represents a random number generation function, and the random number generation function is used to generate a random number between 0 and 1; the Harris hawk fitness function model is: ; where represents the position vector i, is the fitness value corresponding to the position vector i, is a preset adjustment parameter value, is based on the accuracy rate of the indication vector i, represents the total number of preset values in the indication vector i, represents the total number of the multiple candidate features; Through the feature indication function model, convert the target position vector into a target indication vector, where the target indication vector includes the indication values corresponding to each of the multiple candidate features, and when the indication value is a preset value, it is used to represent the selection of the corresponding candidate feature; Select the candidate feature corresponding to the preset value among the multiple candidate features as the target feature.

2. The feature selection method according to claim 1, characterized in that, The updating the position vector i of the Harris hawk to obtain the position vector (i + 1) of the Harris hawk includes: Obtain the escape probability; Based on the escape probability, i, and the preset number of iterations, determine the escape energy; Based on the escape probability and the escape energy, determine the update strategy; Based on the update strategy, update the position vector i of the Harris hawk to obtain the position vector (i + 1) of the Harris hawk.

3. The feature selection method according to claim 2, characterized in that, The determining the escape energy based on the escape probability, i, and the preset number of iterations includes: By , processing the escape probability, i, and the preset number of iterations to obtain the escape energy; Among them, represents the escape energy, and f represents the escape probability, represents the preset number of iterations.

4. A feature selection device, characterized in that, Including: An acquisition module, configured to obtain the initial position vectors of each Harris hawk based on the total number of multiple candidate features in the target dataset, where the initial position vectors include the positions corresponding to the total number; wherein, the target dataset includes CIFAR10; A determination module, configured to determine a target position vector based on a feature indication function model, a Harris hawk fitness function model, and the initial position vectors of the Harris hawks; the determination of the target position vector based on the feature indication function model, the Harris hawk fitness function model, and the initial position vectors of the Harris hawks includes: performing the following operations for the i-th iteration: obtaining the position vectors i of the Harris hawks; converting the position vectors i of the Harris hawks into indication vectors i of the Harris hawks through the feature indication function model; determining the accuracy rates of the indication vectors i of the Harris hawks; processing the accuracy rates of the indication vectors i of the Harris hawks through the Harris hawk fitness function model to obtain the fitness values corresponding to the position vectors i of the Harris hawks; determining the position vector with the minimum fitness value among the position vectors i and the candidate position vector (i - 1) as the candidate position vector i; updating the position vectors i of the Harris hawks to obtain the position vectors (i + 1) of the Harris hawks; incrementing i by 1, and repeating the above operations until, when i is equal to a preset number of iterations, determining the candidate position vector i as the target position vector; initially, i is equal to 1, and the position vector i of the Harris hawk is the initial position vector of the Harris hawk; the feature indication function model is: ; where represents the target position vector, represents the -th position in , represents the probability that the candidate feature corresponding to is selected, represents a random number generation function, and the random number generation function is used to generate a random number between 0 and 1; the Harris hawk fitness function model is: ; where represents the position vector i, is the fitness value corresponding to the position vector i, is a preset adjustment parameter value, is based on the accuracy rate of the indication vector i, represents the total number of preset values in the indication vector i, represents the total number of the multiple candidate features; A conversion module, configured to convert the target position vector into a target indication vector through the feature indication function model, where the target indication vector includes the indication values corresponding to each of the multiple candidate features, and when the indication value is a preset value, it is used to represent the selection of the corresponding candidate feature; A selection module, configured to select the candidate feature corresponding to the preset value among the multiple candidate features as the target feature.

5. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the feature selection method according to any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the feature selection method according to any one of claims 1 to 3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the feature selection method according to any one of claims 1 to 3.

Images