Multi-party high-dimensional unbalanced federal evolution feature selection method

Abstract

The invention relates to the technical field of high-dimensional feature selection, and provides a multi-party high-dimensional unbalanced federal evolution feature selection method, which comprises the following steps that: firstly, a plurality of participants jointly train a gaan network through federal learning, each participant uploads parameters to a federal server for aggregation after convergence, and then the server returns new parameters to each participant, so that a multi-party high-dimensional unbalanced federal evolution feature selection is realized; and updating a local ga parameter. Therefore, the global gaan network can be better trained by combining the data of the participants. According to the data imbalance problem existing in each participant, virtual samples are generated through the trained global ga network for completion, so that the data of each participant is balanced, then an agent model is used for evaluating the particle fitness of each participant, and an evolutionary algorithm is used for iterating a population, so that the particle fitness of each participant is evaluated. In the iteration process, the most characteristic subsets of the participants can be uploaded to a federated server, and the server carries out operation to obtain the subset with the optimal characteristic after aggregation of the participants and returns the subset to the participants.

Description

Technical Field

[0001] The present invention relates to the technical field of high-dimensional feature selection, and more specifically, to a multi-party high-dimensional unbalanced federated evolutionary feature selection method. Background Art

[0002] With the improvement of big data and computing power, significant progress has been made in feature selection methods for high-dimensional unbalanced data both at home and abroad. Early research focused on the classification of feature selection methods, such as filter, wrapper, and embedded methods. With the continuous development of technology, domestic scholars have gradually paid attention to the problem of high-dimensional data imbalance and proposed feature selection methods based on algorithms such as mutual information and particle swarm optimization (PSO), thereby improving the accuracy and generalization ability of classification algorithms.

[0003] With the explosive growth of data scale, the improvement of computing power, and the complexity of high-dimensional data processing, feature selection has become particularly important. Especially in high-dimensional unbalanced small-sample data, how to effectively perform feature selection, not only reducing the data dimension but also considering the data imbalance, poses higher requirements for the accuracy, stability, and generalization ability of classification algorithms.

[0004] In recent years, the combination of federated learning and generative adversarial networks (GANs) has become an effective means to solve the problems of data privacy protection and multi-party data cooperation. As a distributed machine learning method, federated learning protects data privacy by dispersing the training process to each device, where each device only shares model parameters rather than data. Particle swarm optimization (PSO), as a meta-heuristic algorithm, has gradually shown its advantages in feature selection problems and can optimize feature subsets according to the global search ability of the particle swarm. However, challenges such as the "curse of dimensionality", high computational cost, and overfitting have always existed in high-dimensional small-sample feature selection problems.

[19] With the rapid development of edge computing, high-dimensional small-sample data describing the same learning task may be distributed and stored in different institutions (referred to as parties) in reality, and due to privacy protection restrictions, these data cannot be shared among parties. How to select the best-performing feature subset while ensuring the data privacy of parties is an important and extremely challenging topic. Further, some parties may have a lack of certain types of samples, that is, the case of empty classes, which further increases the difficulty of dealing with such problems. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a multi-party high-dimensional unbalanced federated evolutionary feature selection method. First, in the framework of the federated learning system, virtual samples are generated by constructing a generative adversarial network to fill the minority classes or empty classes on each participant while ensuring data privacy. Subsequently, a proxy-assisted federated evolutionary feature selection method is designed by using a proxy model to assist in evaluating the true objective function to solve the high computational cost problem of high-dimensional data.

[0006] To achieve the above object, the present invention provides the following technical solution: A multi-party high-dimensional unbalanced federated evolutionary feature selection method, comprising the following steps:

[0007] Step S1, construct a global generative adversarial network model under the federated learning framework. Among them, each participant first trains a generative adversarial network locally based on its own data. After the local generative adversarial network training converges, each participant uploads the trained network parameters to the federated server for aggregation. The server will send the aggregated parameters back to each participant to update the parameters of its local generative adversarial network. Through multiple rounds of iterative training, a well-performing global generative adversarial network is obtained;

[0008] Step S2, the server sends the aggregated global model parameters back to each participant to update the parameters of the local generative adversarial network of each participant; by introducing the maximum mean discrepancy mechanism and combining the local states of each participant, the global model can better train the global generative adversarial network;

[0009] Step S3, use the trained global generative adversarial network to generate virtual samples for each participant to solve the data imbalance problem. Among them, the generator generates minority class samples according to random noise, and the discriminator evaluates the authenticity of the generated samples. Through adversarial training, the quality of the generated samples is continuously improved, and finally the data distributions of each participant tend to be balanced;

[0010] Step S4, after the data is balanced, each participant uses a proxy model to evaluate the fitness of the particle swarm, and uses an evolutionary algorithm to iteratively optimize the population. During the iteration, each participant uploads the obtained optimal feature subset to the federated server;

[0011] Step S5, the federated server performs operations and aggregations on the received multiple optimal feature subsets to obtain a globally optimal feature subset, and sends the information of this feature subset back to each participant;

[0012] Step S6, each participant receives the information of the globally optimal feature subset, and performs dimensionality reduction processing on the local data based on this feature subset to effectively reduce the data dimension and improve the model training efficiency.

[0013] As a further solution of the present invention, the process of the multiple parties locally training a generative adversarial network includes: constructing a generator capable of generating minority class samples, where the generator includes a neural network structure that receives random noise or vectors in the latent space as inputs and generates synthetic minority class samples; and simultaneously constructing a discriminator, where the discriminator includes a neural network structure for distinguishing real samples from the generated synthetic samples, and the discriminator inputs sample data and outputs the probability that the sample is a real sample.

[0014] As a further solution of the present invention, when the federated server aggregates the generative adversarial network parameters uploaded by the parties, it uses weighted averaging to integrate the model parameters from different parties and distributes the aggregated parameters to each party for updating their local generative adversarial network, and through multiple rounds of iterative training, a global generative adversarial network model with good performance is obtained.

[0015] As a further solution of the present invention, the maximum mean discrepancy mechanism is adopted to evaluate the local models of each party during the training of the global generative adversarial network, and the parameter update strategy is adjusted by calculating the distribution difference between the local model and the global model to ensure that the global model can better integrate the data characteristics of each party.

[0016] As a further solution of the present invention, during the process of using the surrogate model to evaluate the particle fitness, first, feature selection is performed on the local data of each party. By initializing the positions and velocities of the particle swarm, where each particle represents a feature subset, the fitness of each particle is evaluated, and the positions and velocities of the particles are updated through iterative optimization.

[0017] As a further solution of the present invention, after the federated server receives the optimal feature subsets uploaded by each party, by analyzing the commonalities and differences of these feature subsets and combining the data characteristics of each party, a global optimal feature subset that can meet the needs of all parties is comprehensively obtained.

[0018] As a further solution of the present invention, after each party receives the global optimal feature subset, it performs dimensionality reduction processing on the local data based on this feature subset and uses the dimensionally reduced data for subsequent model training, thereby significantly reducing the computational and communication overhead while ensuring the model performance.

[0019] A computer device for a multi-party high-dimensional imbalanced federated evolutionary feature selection method includes: a processor, a memory, and a communication interface, where the communication interface is used to implement parameter interaction between parties and between parties and the federated server; a computer program is stored in the memory, and the computer program, when executed by the processor, is used to implement the training of a generative adversarial network and the feature selection process based on the federated learning framework.

[0020] A computer-readable storage medium for a multi-party high-dimensional unbalanced federated evolutionary feature selection method, on which a computer program for performing the following functions is stored: for calling a processor to implement the training of the participant's local generative adversarial network, the uploading and updating of parameters, the fitness evaluation of the proxy model, and the data dimensionality reduction processing based on the globally optimal feature subset.

[0021] Compared with the prior art, the beneficial effects of this method are as follows: This method uses the federated learning framework for distributed feature selection. Different from the method in the prior art that usually needs to centralize all data to a server for processing, this method shares and aggregates the data parameters of each participant, ensuring data privacy protection and enabling collaborative feature selection in a multi-party distributed environment. This approach not only avoids the data privacy leakage problem in the traditional centralized processing method but also effectively reduces the cost of data transmission and improves the computing efficiency. In contrast, most of the existing methods cannot balance privacy protection and efficient feature selection, often requiring centralized data processing, resulting in computational burdens and privacy risks.

[0022] Compared with the prior art, the beneficial effects of this method are as follows: This method uses a generative adversarial network (GAN) to generate virtual samples to solve the problem of minority class samples in high-dimensional unbalanced data. Traditional feature selection methods often ignore data imbalance, resulting in the inability to effectively solve the situation of insufficient minority class samples, thus affecting the performance of classification algorithms. In contrast, this method generates synthetic minority class samples through GAN, optimizing the data distribution and making the data more balanced, thereby effectively improving the accuracy and generalization ability of classification algorithms. In this way, not only the curse of dimensionality of high-dimensional data is solved, but also the overfitting problem that may occur in traditional methods is avoided, further improving the stability and accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a technical roadmap of a multi-party high-dimensional unbalanced federated evolutionary feature selection method of the present invention.

[0024] Figure 2 It is a schematic diagram of the feature selection optimization process based on the evolutionary algorithm of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] Example 1

[0027] A multi - party high - dimensional unbalanced federated evolutionary feature selection method includes the following steps:

[0028] Step S1, construct a global generative adversarial network model under the federated learning framework. Among them, multiple participants first train the generative adversarial network locally based on their respective data. After the local generative adversarial network training converges, each participant uploads the trained network parameters to the federated server for aggregation. The server then sends the aggregated parameters back to each participant to update their local generative adversarial network parameters. Through multiple rounds of iterative training, a well - performing global generative adversarial network is obtained.

[0029] Step S2, the server sends the aggregated global model parameters back to each participant to update the parameters of the local generative adversarial network of each participant. By introducing the maximum mean discrepancy mechanism and combining the local states of each participant, the global model can better train the global generative adversarial network.

[0030] Step S3, use the trained global generative adversarial network to generate virtual samples for each participant to solve the data imbalance problem. Among them, the generator generates minority - class samples according to random noise, and the discriminator evaluates the authenticity of the generated samples. Through adversarial training, the quality of the generated samples is continuously improved, and finally the data distributions of all participants tend to be balanced.

[0031] Step S4, after the data is balanced, each participant uses a proxy model to evaluate the fitness of the particle swarm and uses an evolutionary algorithm to iteratively optimize the population. During the iteration process, each participant uploads the obtained optimal feature subset to the federated server.

[0032] Step S5, the federated server performs operations and aggregations on the multiple optimal feature subsets received to obtain a globally optimal feature subset, and sends the information of this feature subset back to each participant.

[0033] Step S6, each participant receives the information of the globally optimal feature subset and performs dimensionality reduction processing on the local data based on this feature subset, achieving effective reduction of the data dimension and improvement of the model training efficiency.

[0034] The process of local training of the generative adversarial network by the multiple participants in the embodiment of the present invention includes: constructing a generator capable of generating minority - class samples. The generator includes a neural network structure that receives random noise or vectors in the latent space as inputs and generates synthetic minority - class samples. At the same time, construct a discriminator. The discriminator includes a neural network structure for distinguishing real samples from generated synthetic samples. The discriminator inputs sample data and outputs the probability that the sample is a real sample.

[0035] When solving data imbalance through a generative adversarial network, the generator fills the data of each party with synthesized minority-class samples, optimizes the data distribution, makes the dataset more balanced, and thus improves the accuracy and generalization ability of the classification algorithm.

[0036] In the embodiments of the present invention, when the federated server aggregates the parameters of the generative adversarial network uploaded by the parties, it uses weighted averaging to integrate the model parameters from different parties, and distributes the aggregated parameters to each party for updating its local generative adversarial network. Through multiple rounds of iterative training, a global generative adversarial network model with good performance is obtained.

[0037] In the embodiments of the present invention, when training the global generative adversarial network, the maximum mean discrepancy mechanism is used to evaluate the local models of each party. By calculating the distribution difference between the local model and the global model, the parameter update strategy is adjusted to ensure that the global model can better integrate the data characteristics of each party.

[0038] In the process of using the proxy model to evaluate the particle fitness in the embodiments of the present invention, first, feature selection is performed on the local data of each party. By initializing the positions and velocities of the particle swarm, where each particle represents a feature subset, the fitness of each particle is evaluated, and the positions and velocities of the particles are updated through iterative optimization.

[0039] In the embodiments of the present invention, after the federated server receives the optimal feature subsets uploaded by each party, by analyzing the commonalities and differences of these feature subsets and combining the data characteristics of each party, a global optimal feature subset that can meet the needs of all parties is comprehensively obtained.

[0040] In the embodiments of the present invention, after each party receives the global optimal feature subset, it performs dimensionality reduction processing on the local data based on this feature subset, and uses the dimensionality-reduced data for subsequent model training, thereby significantly reducing the computational and communication overhead while ensuring the model performance.

[0041] Embodiment 2

[0042] In the embodiments of the present invention, the algorithm is implemented using the Matlab programming language. The experimental hardware configuration is an Intel i5 processor, and the operating system is Windows 11. Ten groups of publicly available high-dimensional datasets are selected, and the dimensions of the features range from 2000 to 20000. All datasets are standardized, and the feature values are normalized to the interval from 0 to 1 to reduce the influence of different dimensions. The evaluation metrics used include ACC (classification accuracy), F1-Score (balanced performance metric), the number of feature selections, and the running time. A comparison is made with classical feature selection algorithms such as BPSO, FG-HFS, and BEGJO, and the results are shown in Table 1:

[0043] Table 1

[0044]

[0045]

[0046] As an advanced distributed machine learning method, federated learning allows multiple participants to share the results of model training while keeping the data local. This framework avoids the centralized storage and processing of data, thus reducing the risk of data leakage. The PSO algorithm simulates the foraging behavior of bird flocks and dynamically adjusts the search direction through information exchange and cooperation among particles to achieve the purpose of optimization. Its basic principle is simple and easy to understand, and the algorithm implementation is efficient and easy to program.

[0047] GAN networks, as deep learning models, are widely used in data generation such as images and have demonstrated excellent application potential in multiple fields. With the continuous progress of technology and in-depth research, GAN networks will drive innovation and applications in more fields. Federated learning has shown broad application prospects in multiple industries, such as finance, healthcare, and smart homes. In the financial industry, institutions can share model learning experiences and improve risk assessment capabilities while protecting user privacy; in the healthcare field, hospitals can jointly train diagnostic models to improve the accuracy of disease prediction. The PSO algorithm has been widely applied in function optimization, neural network training, fuzzy system control, and other application fields of genetic algorithms. For example, in function optimization, the PSO algorithm can be used to solve complex non-linear optimization problems; in neural network training, the PSO algorithm can be used to optimize parameters such as weights and biases of neural networks; in fuzzy system control, the PSO algorithm can be used to optimize rules and parameters of fuzzy controllers.

[0048] As Figure 1 shown, in the embodiments of the present invention, to address the problems of data imbalance and lack of classes in local client data, a GAN network is used to generate virtual data.

[0049] The generation process of the GAN network includes:

[0050] Preprocessing: Preprocess the original data set, including steps such as data cleaning and normalization, to ensure the quality and consistency of the data.

[0051] Model construction: Generator: Construct a generator that can generate minority class samples. The generator is usually a neural network whose input is random noise or a vector in the latent space, and the output is the synthesized minority class samples; Discriminator: Construct a discriminator used to distinguish real samples from the synthesized samples generated by the generator. The discriminator is also a neural network whose input is sample data and the output is the probability that the sample is real.

[0052] Adversarial training: Mix real minority-class samples with synthetic samples generated by the generator and then feed them into the discriminator for training.

[0053] In the embodiments of the present invention, the goal of the discriminator is to accurately identify real samples and synthetic samples as much as possible. The goal of the generator is to generate synthetic samples that are realistic enough to make it difficult for the discriminator to distinguish. By alternately training the generator and the discriminator, both are continuously optimized in the confrontation with each other, and finally the generator can generate highly realistic minority-class samples.

[0054] In the embodiments of the present invention, federated learning is used to address the problem of data privacy leakage. Federated learning is a distributed machine learning framework aimed at enabling multiple participating parties to collaboratively train a global model without sharing data. First, multiple participating parties jointly train a GAN network through federated learning. After each participating party converges, it uploads the parameters to the federated server for aggregation. Then, the server sends the new parameters back to each participating party and updates the local GAN parameters. By adding the MMD score, each local model updates according to its own state, thereby better training the global GAN network by combining the data of each participating party. By combining federated learning with the GAN network, the local GAN can learn global information.

[0055] In the embodiments of the present invention, the PSO evolutionary algorithm is used to reduce the dimension of the data to obtain an optimal feature subset and reduce the computational cost. The inspiration for PSO comes from the social behavior of some biological organisms, especially the ability of some animal species to locate ideal positions in a given area. In PSO, a swarm consists of a group of particles, and each particle represents a potential solution to the optimization problem. In each iteration, each particle adjusts its velocity and position by tracking two best positions.

[0056] Figure 2 To show the optimization process of feature selection based on the evolutionary algorithm in the embodiments of the present invention. In the figure, the horizontal axis represents the feature dimension, which represents the search space projection in the feature selection process and reflects the search trajectory and convergence process of the optimization algorithm in the feature space. The vertical axis represents the fitness value, and the dashed line represents the trend of fitness change during the optimization process. The initial particle swarm gradually evolves towards a better solution through iterative optimization and finally converges to the feature combination with the highest fitness value.

[0057] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

[0058] Finally, the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-party high-dimensional unbalanced federated evolution feature selection method, characterized in that: The following steps are involved: Step S1, constructing a global generative adversarial network model under the federated learning framework, wherein the participants first train the generative adversarial network locally based on their own data. After the local generative adversarial network training converges, each participant uploads the trained network parameters to the federated server for aggregation. The server transmits the aggregated parameters back to each participant for updating its local generative adversarial network parameters. Through multiple rounds of iterative training, a global generative adversarial network with good performance is obtained; Step S2: The server transmits the aggregated global model parameters back to each participant, and updates the parameters of each participant's local generative adversarial network. By introducing the maximum mean difference mechanism and combining the local states of each participant, the global model can better train the global generative adversarial network. Step S3, using the trained global generative adversarial network to generate virtual samples for each participant to solve the data imbalance problem, where the generator generates minority class samples based on random noise, and the discriminator evaluates the authenticity of the generated samples. The quality of the generated samples is continuously improved through adversarial training, and finally the data distribution of each participant tends to be balanced; Step S4: After the data is balanced, each participant uses the proxy model to evaluate the fitness of the particle swarm and uses the evolutionary algorithm to iteratively optimize the swarm. During the iteration, the optimal feature subset obtained by each participant is uploaded to the federated server. Step S5: the federated server calculates and aggregates the received multiple optimal feature subsets to obtain a global optimal feature subset, and transmits the feature subset information back to each participant; In step S6, each participant receives the global optimal feature subset information, and performs dimensionality reduction processing on the local data based on the feature subset, thereby effectively reducing the data dimension and improving the model training efficiency.

2. A multi-party high-dimensional unbalanced federated evolution feature selection method according to claim 1, characterized in that: The process of the participant locally training the generative adversarial network includes: constructing a generator capable of generating minority class samples, the generator including a neural network structure, the generator receiving random noise or a vector of a latent space as input, and generating synthetic minority class samples; and constructing a discriminator at the same time, the discriminator including a neural network structure for distinguishing between real samples and generated synthetic samples, the discriminator inputting sample data and outputting the probability that the sample is a real sample.

3. The multi-party high-dimensional unbalanced federated evolution feature selection method according to claim 1, characterized in that: When the federal server aggregates the generative adversarial network parameters uploaded by the participants, it integrates the model parameters from different participants by weighted averaging, and distributes the aggregated parameters to each participant for updating their local generative adversarial networks. Through iterative training, a global generative adversarial network model with good performance is obtained.

4. According to claim 1, a multi-party high-dimensional unbalanced federated evolution feature selection method is characterized in that ,When training the global generative adversarial network, the maximum mean difference mechanism is ,adopted to evaluate the local model of each participant, and the ,parameter updating strategy is adjusted by calculating the distribution difference ,between the local model and the global model.

5. The multi-party high-dimensional unbalanced federated evolution feature selection method according to claim 1, characterized in that: In the process of evaluating particle fitness using the proxy model, firstly, feature selection is performed on the local data of each participant, and the position and velocity of the particle swarm are initialized, where each particle represents a feature subset, the fitness of each particle is evaluated, and the position and velocity of the particle are updated through iterative optimization.

6. The multi-party high-dimensional unbalanced federated evolution feature selection method according to claim 1, characterized in that: After receiving the optimal feature subsets uploaded by each participant, the federal server analyzes the commonalities and differences of these feature subsets and combines the data characteristics of each participant to derive a global optimal feature subset that can meet the needs of all participants.

7. The multi-party high-dimensional unbalanced federated evolution feature selection method according to claim 1, characterized in that: After receiving the global optimal feature subset, each participant performs dimensionality reduction processing on the local data based on the feature subset and uses the reduced dimensionality data for model training.

8. A computer device for implementing the multi-party high-dimensional unbalanced federated evolution feature selection method according to any one of claims 1 to 7, characterized in that: include: A processor, a memory, and a communication interface, wherein the communication interface is used to implement parameter interaction between participants and between participants and a federal server; The memory stores a computer program, which, when executed by the processor, is used to implement a generative adversarial network training and feature selection process based on a federated learning framework.

9. A computer-readable storage medium for implementing a multi-party high-dimensional unbalanced federated evolution feature selection method as described in any one of claims 1 to 7, wherein the computer-readable storage medium stores a computer program that performs the following functions: for calling a processor to implement the training of a local generative adversarial network of a participant, uploading and updating parameters, evaluating the fitness of an agent model, and processing data dimensionality reduction based on a global optimal feature subset.

Images