A method for constructing a federated evolutionary multi-agent assisted feature selection model

Abstract

The present invention provides a method for constructing a federated evolution multi-agent assisted feature selection model, which belongs to the field of data processing technology, including: collecting data sets; for any participant, performing filtering feature selection using an XGBoost tree model; multiple participants perform joint filtering feature selection by uploading feature position indexes with an importance of 0 to a trusted third-party server; all participants obtain a preliminary reduced feature subset RB; for any participant, initialize the initial model of the regression agent, initialize PSO algorithm related parameters; for any participant, perform a PSO-based agent-assisted federated evolution feature selection method on the obtained feature subset RB; and determine whether the current population of all participants meets the termination condition of the algorithm. The present invention can effectively solve the problem of high evaluation cost when the prior art processes multi-party privacy protection scenarios for joint modeling of high-dimensional data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method for constructing a federated evolutionary multi-agent assisted feature selection model. Background Art

[0002] Evolutionary feature selection algorithm refers to an effective method for dealing with feature selection problems using evolutionary optimization technology. The purpose of feature selection is to select some features from all the features of the data set to form an effective feature subset, so as to optimize the set performance indicators while reducing the learning cost. The types, scales and dimensions of data obtained by people in the real world are growing at an unprecedented rate. The ever-increasing search space brings huge challenges to the feature selection problem, such as "dimensionality curse" and time-consuming model evaluation. Due to the complex correlation between features, traditional feature selection methods are difficult to search for the optimal or suboptimal solution to the problem. As a type of heuristic method, evolutionary algorithm realizes global search by maintaining a population composed of potential solutions between generations. It is an effective way to solve feature selection problems and has been widely used in practical problems such as disease diagnosis, hyperspectral image analysis, IoT intrusion detection and natural language processing. However, in real life, data describing the same learning task are often horizontally distributed and stored in different institutions (called participants), that is, different participants have the same attributes (features) but different sample identification numbers. Furthermore, due to privacy and compliance restrictions, it is difficult to share data from all parties, resulting in a fragmented data state. The existing traditional feature selection algorithms for centralized data storage and distributed feature selection algorithms for data sharing environments are no longer applicable. PSO has the advantages of simple concepts, easy implementation and fast convergence speed.

[0003] Federated Learning is a distributed machine learning paradigm that was originally used to solve the problem of Android mobile phone terminal users updating learning models locally. It was later used to solve data silos, protect data privacy and security, and other issues. The federated system can establish a virtual shared model without sharing the data itself by exchanging model parameters under an encryption mechanism. Since the data itself does not move, there is no possibility of leakage at the data level, thereby protecting the user's data privacy. Although federated learning technology has been successfully applied to problems such as medical image processing, Internet of Vehicles, and multi-UAV systems, some work, such as "Federated Incremental Learning Algorithm and Optimization Based on Information Entropy in Industrial Internet of Things" published in the 47th issue of the journal "Journal of Electronics and Information" in May 2024, and the invention patent CN118153707A "A Federated Learning System, Data Processing Method Based on Federated Learning".

[0004] However, when applying the idea of ​​federated learning to the joint construction of feature selection models by multiple institutions under privacy protection, especially for multi-party high-dimensional data (such as genetic data collected by genetic testing institutions, and hyperspectral image data collected by drones in multi-drone systems), there are still the following problems or deficiencies to be solved: First, the joint initial reduction of high-dimensional feature space, that is, how to perform joint filtering feature selection under the premise of ensuring privacy, so as to jointly reduce the initial high-dimensional search space and improve the operating efficiency of the subsequent federated evolutionary feature selection algorithm, is the first problem to be considered. Secondly, the joint evaluation of the individual fitness values ​​(classification accuracy values) of the participants. Since the proxy model can replace the time-consuming real evaluation process, the present invention adopts an evaluation strategy based on the proxy model for individual evaluation. However, the local proxy models constructed by different participants need to be uploaded to a third-party server through a federated learning mechanism to construct a global proxy model. Therefore, how to jointly construct a global proxy model with multiple local proxy models, how to use the proxy models on different participants to jointly evaluate the pros and cons of potential solutions, and obtain more accurate real evaluation results at a lower computational cost, is the second problem to be considered. Finally, the third issue that needs to be considered is the joint update of the populations of the participants, that is, how to design a population evolution strategy that takes into account multiple participants under a federal framework, so that multiple populations evolve and search in a consistent direction to obtain the optimal feature subset for the system. Summary of the invention

[0005] The purpose of the present invention is to provide a method for constructing a federated evolutionary multi-agent assisted feature selection model, which can effectively solve the problem of high evaluation cost when the existing technology handles joint modeling of high-dimensional data in a privacy protection scenario involving multiple parties.

[0006] To achieve the above object, the present invention provides a method for constructing a federated evolutionary multi-agent assisted feature selection model, comprising the following steps:

[0007] S1, collecting data sets, preprocessing the collected data sets and extracting feature vectors, and then normalizing the feature vectors;

[0008] S2. For any participant, use the XGBoost tree model to perform filtering feature selection to obtain the importance of each feature;

[0009] Multiple parties perform joint filtering feature selection by uploading feature position indexes with importance 0 to a trusted third-party server, denoted as Party A;

[0010] Finally, all participants obtain a preliminarily reduced feature subset RB;

[0011] S3. For any participant, initialize the initial model of the regression agent and initialize the relevant parameters of the PSO algorithm. All participants perform the same operation. The regression agent model can select a radial basis neural network model (also known as an RBF agent model) or a Gaussian process model (also known as a kriging model).

[0012] S4. For any participant, the feature subset RB obtained in step S2 is subjected to the agent-assisted federated evolution feature selection method based on PSO, including the joint construction and management (update) strategy of the agent model, the joint evaluation strategy based on the agent model, and the joint update strategy of the particles. All participants perform the same operation, so that the participants take into account the data of all participants without leaking the privacy of the original data, select the feature subset with the best performance of the current participant, and accelerate the convergence of the system;

[0013] S5. Determine whether the current population of all participants has reached the termination condition of the algorithm. If so, all participants upload the optimal feature subset obtained in step S4 to Party A, which performs the optimal feature subset integration and outputs the system comprehensive optimal feature subset; otherwise, return to step S4 and continue to execute the PSO-based agent-assisted federated evolution feature selection process.

[0014] Preferably, in step S1, the data set includes image data and real number data;

[0015] The preprocessing of image data includes: using superpixel segmentation method on image data, and then extracting the texture, color, brightness and shape features of the image;

[0016] The preprocessing of real data includes: when there is missing data, the missing data is filled by interpolation method;

[0017] The normalization method is as follows:

[0018]

[0019] Among them, X is a certain characteristic value; X max is the maximum value corresponding to this feature; X min is the minimum value corresponding to this feature; X * is the normalized result.

[0020] Preferably, in step S2, the specific operations are:

[0021] S21. For any participant, train the XGBoost tree model. Since the XGBoost tree model can select the feature with the most obvious classification effect for splitting when splitting each node, the participant finds the feature position indexes with all importance levels of 0 by counting the number of times the feature is split on the child nodes;

[0022] All parties perform the same operations;

[0023] S22, all participants transmit the feature position index with importance level 0 to Party A, and Party A determines the intersection information of these index values;

[0024] S23. Party A feeds back the intersection information of the index values ​​to each participant, and each participant deletes all the features corresponding to the intersection information in turn, thereby completing the joint filtering feature selection process. Finally, each participant obtains a preliminary reduced feature subset RB, and its corresponding feature space is the reduced feature space.

[0025] Preferably, in step S2, the XGBoost tree model related parameters, including the maximum depth of the tree max_depth, are set to an integer of 5, 6 or 7 (the system default value is 6), the proportions of random sample sampling and random feature sampling of each tree subsample and colsample_bytree are set to any value in [0.7, 1], and the sum of the minimum leaf node sample weights required for node splitting min_child_weight is set to any value in [0.8, 1] (the system default value is 1).

[0026] Preferably, in step S3, the regression proxy model is any one of a radial basis function neural network model (also known as an RBF proxy model) or a Gaussian process model (also known as a kriging model).

[0027] Preferably, in step S3, the specific operations are:

[0028] S31, for the reduced feature space obtained in step S2, generate N initial sample points (initial particles) by means of Latin hypercube sampling to form the initial population pop of the evolutionary algorithm;

[0029] S32, using the entire training sample set and the classifier to evaluate the true classification accuracy value of each particle, these particles (sample points) and their true classification accuracy values ​​will be used as sampling points and saved in the proxy sample pool SP;

[0030] S33, using the sampling points in the SP to train the proxy model, thereby completing the initial construction of the local regression proxy model on the participant, and obtaining the initial local proxy model;

[0031] S34, initialize PSO algorithm related parameters, including population size popsize set to 30 ~ 50, maximum number of iterations T max Set to 50~100, control parameters c1=rand, c2=1-c1, η=0.2.

[0032] Preferably, in step S4, the specific operations are:

[0033] S41. Assume that any individual X in the initial population is encoded using the probability value obtained by Latin hypercube sampling, and the formula is as follows:

[0034] X=(x1,x2,...,x D ),x i ∈[0,1],i=1,2,...,D (2)

[0035] Where D is the number of all features; x i represents the probability of the i-th feature being selected in an individual. If x i >0.5, it means that the i-th feature is selected into the feature subset and decoded as 1; otherwise, this feature is eliminated and decoded as 0;

[0036] S42, performing joint construction and management of local proxy models;

[0037] First, build a local proxy model;

[0038] Subsequently, the joint management (update) of the local proxy model is performed. When the joint update period τ1 of the proxy model is not reached, the local proxy model update without considering the global information is performed; when the joint update period τ1 of the proxy model is reached, the local proxy model adaptive update considering the global information is performed;

[0039] Finally, an updated local proxy model is obtained;

[0040] S43, performing joint construction and management of a global proxy model;

[0041] First, the local proxy models on different parties are combined to build a global proxy model of the system on party A;

[0042] Subsequently, the global proxy model is updated every τ1 generations;

[0043] Finally, an updated global proxy model is obtained;

[0044] S44, executing the joint evaluation strategy based on the agent model; since the particles only exist on each participating party, the evaluation process is completed on the participating party.

[0045] First, the updated local proxy model obtained in step S42 is used to predict the classification accuracy values ​​of all particles in the current population, and the particles are sorted from large to small according to the classification accuracy values;

[0046] Subsequently, the top 10% of the ranked particles are selected and their true classification accuracy values ​​are evaluated by the classifier KNN;

[0047] Since the local proxy model used to evaluate the current particle may be a global proxy model constructed by uniting all participants, this process embodies a “joint” evaluation process of the local proxy model and the global proxy model.

[0048] S45, executing particle joint update strategy;

[0049] Since it is necessary to take into account the data information of all participants and jointly search for the system's optimal feature subset, it is necessary to periodically jointly update the optimal feature subsets (optimal particles) on different participants.

[0050] The joint update period of particles is τ2. For any participant, first, when the joint update period τ2 is reached, the best particle corresponding to the highest true evaluation value (i.e., the global best particle) is selected from the top 10% of the better particles with true evaluation and transmitted to Party A.

[0051] Then, with party A as the intermediary, the participant receives the optimal feature subsets of the other M-1 participants from party A to replace the poor M-1 particles in the current population;

[0052] Where M represents the number of participants;

[0053] Finally, an updated population is obtained.

[0054] Preferably, in step S42, when the proxy model joint update period τ1 is not reached, first, all particles in the current generation population are evaluated locally, and the evaluation values ​​are sorted from large to small; then, the top 10% of the particles are selected for further real evaluation, which are the better particles in the current population and can guide the evolution direction of the population, which will help to build a more accurate local proxy model; finally, the top 10% of the particles and their real evaluation values ​​are used as new sampling points and added to the proxy sample pool SP to update the local proxy model of the current generation;

[0055] It should be noted that in order to prevent adding more sampling points from affecting the efficiency of proxy model update and reduce the impact of poor quality sampling points on model update, it is necessary to set the size of the proxy sample pool SP. Here, |SP max =2×popsize. When the scale of the sampling points in the proxy sample pool SP exceeds its upper limit, the sampling point with the worst true evaluation value is deleted to meet the scale of the proxy sample pool SP.

[0056] When the joint update period τ1 of the proxy model is reached, the participating party adaptively chooses whether to replace the global proxy model with the current local proxy model by comparing the evaluation errors of the global proxy model and the current local proxy model on some better particles.

[0057] Preferably, in step S43, the specific operations are:

[0058] S431, for any participant, uploading the initially constructed local proxy model parameters to Party A;

[0059] S432. Drawing on the idea of ​​federated learning, the model parameters from different participants are weighted and aggregated, and the result of the aggregation is the initial global proxy model constructed by Party A. Taking the RBF proxy model as an example, the proxy model consists of only one input layer, one hidden layer and one output layer. Its parameters mainly include the weight matrix from the hidden layer to the output layer and the bias vector, denoted as rbf, rbf = [w1, w2, ..., w q ,b], where q is the number of neurons in the hidden layer, w j is the weight vector of the jth neuron, b is the bias vector from the hidden layer to the output layer, then the local proxy model parameter of the i-th participant is expressed as rbf i =[w i1 ,w i2 ,…,w iq ,b i ]. The weighted aggregation formula of party A is as follows:

[0060]

[0061] Since it is impossible to determine the impact of different local proxy models on the global proxy model, the present invention adopts an average aggregation method to construct a global proxy model.

[0062] S433. For any participant, when the proxy model joint update period τ1 is reached, the current generation local proxy model parameters and the mean square error between the local proxy model evaluation value of some particles and the true classifier evaluation value are transmitted to Party A;

[0063] The purpose of transmitting the local proxy model parameters is to construct the global proxy model of the current generation. The parameter update formula is shown in formula (3). The purpose of transmitting the mean square error is to determine whether the current local proxy model needs to be replaced by the global proxy model.

[0064] The mean square error is calculated by the sampling points in the proxy sample pool SP. The current generation of sampling points in the proxy sample pool SP is denoted as Z, where Z = {z1, z2, …, z SP}, where z k =[x k ,f(x k )],k=1,2,…,SP,x k represents the particle corresponding to the kth sampling point, f(x k ) represents x k The true evaluation value of

[0065] For the i-th participant, the mean square error calculation formula is as follows:

[0066]

[0067] in, represents the particle x on the ith participant k The predicted fitness value evaluated by the local proxy model; Party A receives the mean square error of all participants and then calculates the mean of the mean square error, which is recorded as Determine the size of MSE(i) and MMSE. When MSE(i)>MMSE, it means that the error of the local proxy model of the i-th participant is relatively large and needs to be updated, that is, replaced with the current global proxy model; otherwise, it means that the local proxy model of the i-th participant has a high degree of fit with the true classifier and does not need to be updated or replaced.

[0068] Therefore, the present invention adopts the above-mentioned method for constructing a federated evolutionary multi-agent assisted feature selection model, and the technical effects are as follows:

[0069] (1) It solves the problem of high evaluation cost when existing technologies jointly construct feature selection models with multi-party high-dimensional data under privacy protection, and improves computational efficiency;

[0070] (2) It can be applied to the joint feature selection modeling of multiple genetic testing institutions. Under the premise of ensuring the privacy and security of genetic data from multiple parties, it can not only improve the classification accuracy of the model to a certain extent, but also greatly reduce the cost of modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A basic framework diagram of a method for constructing a federated evolutionary multi-agent assisted feature selection model according to the present invention;

[0072] Figure 2 Taking the interaction between the first participant and the trusted third party (Party A) as an example, a specific framework diagram of the method proposed in the present invention is given;

[0073] Figure 3 The method proposed in this invention is used on three commonly used data sets (Musk1, Isolet and BASEHOCK) to compare the results obtained without adding a proxy model, using the Kriging model and using the RBF proxy model; Figure 3 (a) in the figure is the comparison of classification accuracy results; Figure 3 (b) in the figure is the comparison of running time results. DETAILED DESCRIPTION

[0074] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0075] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0076] Embodiment 1

[0077] like Figure 1 As shown, it is a basic framework diagram of a method for constructing a federated evolutionary multi-agent assisted feature selection model of the present invention ( Figure 1 In the above, ① represents the feature position index value whose number of splits is 0; ② represents the feature position index value whose number of splits of all participants is 0 at the same time; ③ represents the local proxy model parameters and the mean square error value of the SP sampling points on each participant; ④ represents the global proxy model parameters).

[0078] Figure 2 Taking the interaction between the first participant and the trusted third party (Party A) as an example, a specific framework diagram of the method proposed in the present invention is given, which specifically includes the following steps:

[0079] S1, collecting data sets, preprocessing the collected data sets and extracting feature vectors, and then normalizing the feature vectors;

[0080] The dataset includes image data and real number data;

[0081] The preprocessing of image data includes: using superpixel segmentation method on image data, and then extracting the texture, color, brightness and shape features of the image;

[0082] The preprocessing of real data includes: when there is missing data, the missing data is filled by interpolation method;

[0083] The normalization method is as follows:

[0084]

[0085] Among them, X is a certain characteristic value; X max is the maximum value corresponding to this feature; X min is the minimum value corresponding to this feature; X * is the normalized result.

[0086] S2. For any participant, use the XGBoost tree model to perform filtering feature selection to obtain the importance of each feature;

[0087] Multiple parties perform joint filtering feature selection by uploading feature position indexes with importance 0 to a trusted third-party server, denoted as Party A;

[0088] Finally, all participants obtain a preliminarily reduced feature subset RB;

[0089] Specific:

[0090] S21. For any participant, train the XGBoost tree model. Since the XGBoost tree model can select the feature with the most obvious classification effect for splitting when splitting each node, the participant finds the feature position indexes with all importance levels of 0 by counting the number of times the feature is split on the child nodes;

[0091] All parties perform the same operations;

[0092] XGBoost tree model related parameters, including the maximum tree depth max_depth set to an integer 5, 6 or 7 (the system default value is 6), the proportion of random sample sampling and random feature sampling for each tree subsample and colsample_bytree are set to any value in [0.7, 1], and the minimum sum of leaf node sample weights required for node splitting min_child_weight is set to any value in [0.8, 1] (the system default value is 1).

[0093] S22, all participants transmit the feature position index with importance level 0 to Party A, and Party A determines the intersection information of these index values;

[0094] S23. Party A feeds back the intersection information of the index values ​​to each participant, and each participant deletes all the features corresponding to the intersection information in turn, thereby completing the joint filtering feature selection process. Finally, each participant obtains a preliminary reduced feature subset RB, and its corresponding feature space is the reduced feature space.

[0095] S3. For any participant, initialize the initial model of the regression agent and initialize the relevant parameters of the PSO algorithm. All participants perform the same operation. The regression agent model can be a radial basis neural network model (also known as an RBF agent model), a Gaussian process model (also known as a kriging model), a support vector regression model (also known as an SVR model), and a polynomial regression model (also known as a PR model).

[0096] Specific:

[0097] S31, for the reduced feature space obtained in step S2, generate N initial sample points (initial particles) by means of Latin hypercube sampling to form the initial population pop of the evolutionary algorithm;

[0098] S32, using the entire training sample set and the classifier to evaluate the true classification accuracy value of each particle, these particles (sample points) and their true classification accuracy values ​​will be used as sampling points and saved in the proxy sample pool SP;

[0099] S33, using the sampling points in the SP to train the proxy model, thereby completing the initial construction of the local regression proxy model on the participant, and obtaining the initial local proxy model;

[0100] S34, initialize PSO algorithm related parameters, including population size popsize set to 30 ~ 50, maximum number of iterations T max Set to 50~100, control parameters c1=rand, c2=1-c1, η=0.2.

[0101] S4. For any participant, the feature subset RB obtained in step S2 is subjected to the agent-assisted federated evolution feature selection method based on PSO, including the joint construction and management (update) strategy of the agent model, the joint evaluation strategy based on the agent model, and the joint update strategy of the particles. All participants perform the same operation, so that the participants take into account the data of all participants without leaking the privacy of the original data, select the feature subset with the best performance of the current participant, and accelerate the convergence of the system;

[0102] S41. Assume that any individual X in the initial population is encoded using the probability value obtained by Latin hypercube sampling, and the formula is as follows:

[0103] X=(x1,x2,...,x D ),x i ∈[0,1],i=1,2,...,D (2)

[0104] Where D is the number of all features; x i represents the probability of the i-th feature being selected in an individual. If x i >0.5, it means that the i-th feature is selected into the feature subset and decoded as 1; otherwise, this feature is eliminated and decoded as 0;

[0105] S42, performing joint construction and management of local proxy models;

[0106] First, build a local proxy model;

[0107] Subsequently, the joint management (update) of the local proxy model is performed. When the joint update period τ1 of the proxy model is not reached, the local proxy model update without considering the global information is performed; when the joint update period τ1 of the proxy model is reached, the local proxy model adaptive update considering the global information is performed;

[0108] Finally, an updated local proxy model is obtained;

[0109] When the proxy model joint update cycle τ1 is not reached, first, all particles in the current generation population are evaluated locally, and the evaluation values ​​are sorted from large to small; then, the top 10% of the particles are selected for further real evaluation. These particles are the better particles in the current population and can guide the evolution direction of the population, which will help build a more accurate local proxy model; finally, the top 10% of the particles and their real evaluation values ​​are used as new sampling points and added to the proxy sample pool SP to update the local proxy model of the current generation;

[0110] It should be noted that in order to prevent adding more sampling points from affecting the efficiency of proxy model update and reduce the impact of poor quality sampling points on model update, it is necessary to set the size of the proxy sample pool SP. Here, |SP max =2×popsize. When the scale of the sampling points in the proxy sample pool SP exceeds its upper limit, the sampling point with the worst true evaluation value is deleted to meet the scale of the proxy sample pool SP.

[0111] When the joint update period τ1 of the proxy model is reached, the participating party adaptively chooses whether to replace the global proxy model with the current local proxy model by comparing the evaluation errors of the global proxy model and the current local proxy model on some better particles.

[0112] S43, performing joint construction and management of a global proxy model;

[0113] First, the local proxy models on different parties are combined to build a global proxy model of the system on party A;

[0114] Subsequently, the global proxy model is updated every τ1 generations;

[0115] Finally, an updated global proxy model is obtained;

[0116] S431, for any participant, uploading the initially constructed local proxy model parameters to Party A;

[0117] S432. Drawing on the idea of ​​federated learning, the model parameters from different participants are weighted and aggregated, and the result of the aggregation is the initial global proxy model constructed by Party A. Taking the RBF proxy model as an example, the proxy model consists of only one input layer, one hidden layer and one output layer. Its parameters mainly include the weight matrix from the hidden layer to the output layer and the bias vector, denoted as rbf, rbf = [w1, w2, ..., w q ,b], where q is the number of neurons in the hidden layer, w j is the weight vector of the jth neuron, b is the bias vector from the hidden layer to the output layer, then the local proxy model parameter of the i-th participant is expressed as rbf i =[wi1 ,w i2 ,…,w iq ,b i ]. The weighted aggregation formula of party A is as follows:

[0118]

[0119] Since it is impossible to determine the impact of different local proxy models on the global proxy model, the present invention adopts an average aggregation method to construct a global proxy model.

[0120] S433. For any participant, when the proxy model joint update period τ1 is reached, the current generation local proxy model parameters and the mean square error between the local proxy model evaluation value of some particles and the true classifier evaluation value are transmitted to Party A;

[0121] The purpose of transmitting the local proxy model parameters is to construct the global proxy model of the current generation. The parameter update formula is shown in formula (3). The purpose of transmitting the mean square error is to determine whether the current local proxy model needs to be replaced by the global proxy model.

[0122] The mean square error is calculated by the sampling points in the proxy sample pool SP. The current generation of sampling points in the proxy sample pool SP is denoted as Z, where Z = {z1, z2, …, z SP}, where z k =[x k ,f(x k )],k=1,2,…,SP,x k represents the particle corresponding to the kth sampling point, f(x k ) represents x k The true evaluation value of

[0123] For the i-th participant, the mean square error calculation formula is as follows:

[0124]

[0125] in, represents the particle x on the ith participant k The predicted fitness value evaluated by the local proxy model; Party A receives the mean square error of all participants and then calculates the mean of the mean square error, which is recorded as Determine the size of MSE(i) and MMSE. When MSE(i)>MMSE, it means that the error of the local proxy model of the i-th participant is relatively large and needs to be updated, that is, replaced with the current global proxy model; otherwise, it means that the local proxy model of the i-th participant has a high degree of fit with the true classifier and does not need to be updated or replaced.

[0126] S44, executing the joint evaluation strategy based on the agent model; since the particles only exist on each participating party, the evaluation process is completed on the participating party.

[0127] First, the updated local proxy model obtained in step S42 is used to predict the classification accuracy values ​​of all particles in the current population, and the particles are sorted from large to small according to the classification accuracy values;

[0128] Subsequently, the top 10% of the ranked particles are selected and their true classification accuracy values ​​are evaluated by the classifier KNN;

[0129] Since the local proxy model used to evaluate the current particle may be a global proxy model constructed by uniting all participants, this process embodies a “joint” evaluation process of the local proxy model and the global proxy model.

[0130] S45, executing particle joint update strategy;

[0131] Since it is necessary to take into account the data information of all participants and jointly search for the system's optimal feature subset, it is necessary to periodically jointly update the optimal feature subsets (optimal particles) on different participants.

[0132] The joint update period of particles is τ2. For any participant, first, when the joint update period τ2 is reached, the best particle corresponding to the highest true evaluation value (i.e., the global best particle) is selected from the top 10% of the better particles with true evaluation and transmitted to Party A.

[0133] Then, with party A as the intermediary, the participant receives the optimal feature subsets of the other M-1 participants from party A to replace the poor M-1 particles in the current population;

[0134] Where M represents the number of participants;

[0135] Finally, an updated population is obtained.

[0136] S5. Determine whether the current population of all participants has reached the termination condition of the algorithm. If so, all participants upload the optimal feature subset obtained in step S4 to Party A, which performs the optimal feature subset integration and outputs the system comprehensive optimal feature subset; otherwise, return to step S4 and continue to execute the PSO-based agent-assisted federated evolution feature selection process.

[0137] The method of the present invention is described below by means of specific examples.

[0138] The datasets used are from the UCI database of chemistry (Musk1), speech recognition (Isolet), and text recognition (BASEHOCK).

[0139] Among them, the Musk1 dataset contains 166 features and 476 test samples, which are divided into 2 categories; the Isolet dataset contains 617 features and 1560 test samples, which are divided into 26 categories; the BASEHOCK dataset contains 4862 features and 1993 test samples, which are divided into 2 categories. Without loss of generality, all datasets are divided into three parties for discussion, that is, any dataset is randomly and evenly divided and randomly assigned to three parties.

[0140] The comparison algorithm used is a multi-surrogate-assisted dual-layer ensemble feature selection algorithm. The algorithm uses a two-layer hybrid ensemble feature selection framework. First, three representative samples are selected by three clustering methods, and PSO is used on each representative sample to obtain three optimal feature subsets. Then, the feature subsets are integrated by using intersection, union and voting methods respectively. Finally, the best one among the six feature subsets is selected as the final result, which can greatly reduce the size of the selected feature subset while improving the classification accuracy. The algorithm is embedded in the encapsulated ensemble feature selection algorithm framework as a comparison algorithm. The reasons for this are: first, there is currently no federated evolutionary feature selection method for multi-party high-dimensional data sets; second, the encapsulated ensemble framework uses a trusted third party to integrate the results obtained by each participant, which can be directly extended to the privacy-preserving feature selection problem.

[0141] To ensure the fairness of the comparison, the maximum number of iterations of the population is T max are all set to 100, and the population size is 50. The parameters selected in this embodiment are as follows: the maximum depth of the tree max_depth = 10; the proportion of random sample sampling of each tree subsample = 0.9; the proportion of random sample sampling and random feature sampling of each tree colsample_bytree = 0.9; the sum of the minimum leaf node sample weights required for node splitting min_child_weight = 1; the joint update cycle of the rational model τ1 = 20; the joint update cycle of particles τ2 = 30. The remaining parameters of the comparison algorithm refer to the settings in the study of Jiang et al. in 2021. To avoid feature selection bias, all algorithms use 10-fold cross validation to calculate the classification accuracy of each candidate solution. Each algorithm was run independently 30 times, and their average optimization results were counted. All algorithms were programmed in Matlab and run on a PC with AMD Ryzen75800H, 3.2GHz and 16GB RAM. Table 1 shows the classification accuracy values ​​and running times obtained by the method proposed in the present invention and the comparison method on the above three types of data sets.

[0142] In addition, in order to verify whether the addition of the proxy model improves the performance of the algorithm, the non-proxy model (None), Kriging model and RBF proxy model are selected as comparisons for the method proposed in this invention. Figure 3 The average classification accuracy and running time of the method proposed in the present invention under these three conditions are given.

[0143] Table 1 Classification accuracy and running time (in seconds) of the proposed method and the comparison method on three types of data sets

[0144]

[0145] As can be seen from Table 1, the method proposed in the present invention not only has certain advantages in classification accuracy, but also its computational efficiency has been greatly improved. It should be noted that for the Musk1 data set, the running time of the method proposed in the present invention is longer than that of the comparison algorithm. This may be because, when the feature dimension is low, the sample agent based on clustering in the comparison algorithm will quickly obtain the training result, while the time for XGBoost model training and agent joint update in the method proposed in the present invention is relatively long, so the overall time consumption is large. When the feature dimension is high, most of the evaluation processes of the method proposed in the present invention are evaluated using the agent model, which significantly reduces the time consumption of the algorithm. Obviously, the method proposed in the present invention greatly improves the running time on high-dimensional data sets.

[0146] from Figure 3 It can be seen that the running time of adding a proxy model is greatly improved compared to that of not adding a proxy model; although for data sets with lower feature dimensions, adding a proxy model may slightly reduce the classification performance of the algorithm, but as the feature dimension increases, the advantage of the proxy model becomes more obvious.

[0147] Therefore, the present invention adopts the above-mentioned federated evolutionary multi-agent assisted feature selection model construction method, which can effectively solve the problem of high evaluation cost when the existing technology handles joint modeling of high-dimensional data in privacy protection scenarios involving multiple parties.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for constructing a federated evolutionary multi-agent assisted feature selection model, characterized in that: The following steps are involved: S1, collecting data sets, preprocessing the collected data sets and extracting feature vectors, and then normalizing the feature vectors; S2. For any participant, use the XGBoost tree model to perform filtering feature selection to obtain the importance of each feature; Multiple parties perform joint filtering feature selection by uploading feature position indexes with importance 0 to a trusted third-party server, denoted as Party A; Finally, all participants obtain a preliminarily reduced feature subset RB; S3. For any participant, initialize the initial model of the regression agent and the relevant parameters of the PSO algorithm. All participants perform the same operation. S4. For any participant, for the feature subset RB obtained in step S2, perform the PSO-based agent-assisted federated evolution feature selection method, including the joint construction and management strategy of the agent model, the joint evaluation strategy based on the agent model, and the joint update strategy of the particles. All participants perform the same operation to select the feature subset with the best performance of the current participant; S5. Determine whether the current population of all participants has reached the termination condition of the algorithm, which is: the product of the population size and the maximum number of iterations; if so, all participants upload the optimal feature subset obtained in step S4 to party A, and party A performs the optimal feature subset integration and outputs the system comprehensive optimal feature subset; otherwise, return to step S4 and continue to execute the PSO-based agent-assisted federated evolution feature selection process; In step S1, the data set includes image data and real number data; The preprocessing of image data includes: using superpixel segmentation method on image data, and then extracting the texture, color, brightness and shape features of the image; The preprocessing of real data includes: when there is missing data, the missing data is filled by interpolation method; The normalization method is as follows: （1） in, is a certain characteristic value; is the maximum value corresponding to this feature; is the minimum value corresponding to this feature; is the normalized result; In step S2, the specific operations are: S21. For any participant, train the XGBoost tree model. The participant finds all feature position indexes with an importance of 0 by counting the number of feature splits on the child nodes. All parties perform the same operations; S22, all participants transmit the feature position index with importance level 0 to Party A, and Party A determines the intersection information of these index values; S23, Party A feeds back the intersection information of the index values ​​to each participant, and each participant deletes all the features corresponding to the intersection information in turn, thereby completing the joint filtering feature selection process. Finally, each participant obtains a preliminary reduced feature subset RB, and its corresponding feature space is the reduced feature space; In step S4, the specific operations are: S41. Assume that any individual X in the initial population is encoded using the probability value obtained by Latin hypercube sampling, and the formula is as follows: （2） in, is the number of all features; Indicates the individual The probability of a feature being selected is >0.5, it means A feature is selected into the feature subset and decoded as 1; otherwise, this feature is removed and decoded as 0; S42, performing joint construction and joint management of local proxy models; First, build a local proxy model; Subsequently, the joint management of the local proxy model is performed, and when the proxy model joint update cycle is not reached When the proxy model joint update cycle is reached, the local proxy model update without considering the global information is performed; When , an adaptive update of the local proxy model taking into account the global information is performed; Finally, an updated local proxy model is obtained; S43, performing joint construction and management of a global proxy model; S44, executing a joint evaluation strategy based on an agent model; First, the updated local proxy model obtained in step S42 is used to predict the classification accuracy values ​​of all particles in the current population, and the particles are sorted from large to small according to the classification accuracy values; Subsequently, the top 10% of the ranked particles are selected, and their true classification accuracy values ​​are evaluated by the classifier KNN; S45, executing particle joint update strategy; The joint update period of particles is , for any participant, first, when the joint update cycle is reached When , among the top 10% of the better particles with real evaluation, the best particle corresponding to the highest value of real evaluation is selected and transmitted to Party A; Then, with party A as the intermediary, the participant receives the optimal feature subsets of the other M-1 participants from party A to replace the poor M-1 particles in the current population; Where M represents the number of participants; Finally, an updated population is obtained.

2. A method for constructing a federated evolutionary multi-agent assisted feature selection model according to claim 1, characterized in that: In step S2, the XGBoost tree model related parameters, including the maximum tree depth max_depth, are set to an integer of 5, 6 or 7, the random sample sampling subsample of each tree is set to any value in [0.7, 1], the proportion of random feature sampling colsample_bytree is set to any value in [0.7, 1], and the minimum sum of leaf node sample weights required for node splitting min_child_weight is set to any value in [0.8, 1].

3. A method for constructing a federated evolutionary multi-agent assisted feature selection model according to claim 2, characterized in that: In step S3, the regression proxy model is any one of a radial basis neural network model or a Gaussian process model.

4. A method for constructing a federated evolutionary multi-agent assisted feature selection model according to claim 3, characterized in that: In step S3, the specific operations are: S31, for the reduced feature space obtained in step S2, using Latin hypercube sampling to generate N initial particles to form an initial population pop of the evolutionary algorithm; S32, using the entire training sample set and the classifier to evaluate the true classification accuracy value of each particle, these particles and their true classification accuracy values ​​will be used as sampling points and saved in the proxy sample pool SP; S33, using the sampling points in the proxy sample pool SP to train the proxy model, thereby completing the initial construction of the local regression proxy model on the participant, and obtaining the initial local proxy model; S34, initialize PSO algorithm related parameters, including population size popsize Set to 30~50, the maximum number of iterations T max Set to 50~100, control parameters c 1= rand , c 2=1- c 1, =0.

2.

5. A method for constructing a federated evolutionary multi-agent assisted feature selection model according to claim 4, characterized in that: In step S42, when the proxy model joint update period has not been reached When , first, all particles in the current generation population are evaluated locally by proxy, and the evaluation values ​​are sorted from large to small; then, the top 10% of the particles are selected for further real evaluation; finally, the top 10% of the particles and their real evaluation values ​​are used as new sampling points and added to the proxy sample pool SP to update the local proxy model of the current generation; When the proxy model joint update cycle is reached By comparing the evaluation errors of the global proxy model and the current local proxy model on some better particles, the participating party adaptively chooses whether to replace the global proxy model with the current local proxy model.

6. A method for constructing a federated evolutionary multi-agent assisted feature selection model according to claim 5, characterized in that: In step S43, the specific operations are: S431, for any participant, uploading the initially constructed local proxy model parameters to Party A; S432. Drawing on the idea of ​​federated learning, the model parameters from different participants are weighted and aggregated. The result of the aggregation is the initial global proxy model constructed by Party A. S433: For any participant, when the agent model joint update cycle is reached When , the current generation of local proxy model parameters and the mean square error between the local proxy model evaluation value and the true classifier evaluation value of some particles are transmitted to party A; The mean square error is calculated by the sampling points in the proxy sample pool SP. The current generation of sampling points in the proxy sample pool SP is recorded as , ,in, , , Indicates The particles corresponding to the sampling points are express The true evaluation value of For the The formula for calculating the mean square error is as follows: （3） in, Indicates Particles on each party The predicted fitness value evaluated by the local proxy model; Party A receives the mean square error of all participants and then calculates the mean of the mean square error, which is recorded as ;judge and The size of , use the current global proxy model to replace it; otherwise, it does not need to be updated or replaced.

Images