Federal fuzzy system optimization method based on particle information cooperation

Abstract

The invention belongs to the field of intelligent computing, and relates to a federal fuzzy system optimization method based on particle information cooperation. The invention provides a particle information collaboration-based fuzzy system optimization method (PICFFS) aiming at the challenge of finding a global optimal solution on the premise of ensuring data privacy and high calculation efficiency in federal fuzzy learning. According to the method, a double-layer optimization mechanism is innovatively designed: on a client side, a particle swarm optimization algorithm is adopted to carry out parameter search on a local fuzzy rule base, and the limitation that a traditional gradient descent method is prone to falling into local optimum is broken through; on a server side, through a particle distance information federation aggregation strategy, only inter-particle relative position features rather than original data are exchanged so as to realize privacy enhancement. Compared with the prior art, the method has the advantages that the global optimal solution can be quickly found without depending on initial value setting while the data privacy is ensured, and the method shows superiority in a plurality of evaluation indexes.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent computing and relates to an optimization method for a federated fuzzy system based on particle information collaboration. Technical Background

[0002] Federated Learning (FL) has become a hot research direction in the field of machine learning in recent years. With data privacy protection at its core, it enables model training in a distributed environment, avoiding the concentration of raw data on a central server and significantly reducing the risk of data leakage. FL distributes model updates to multiple clients and integrates these updates in a centralized or decentralized manner to achieve the goal of global optimization. This feature has shown broad application prospects in fields with extremely high requirements for data privacy protection, such as healthcare, finance, intelligent devices, and education. For example, in the healthcare field, FL can be used for distributed electronic medical record analysis to train disease prediction models while strictly protecting patient privacy.

[0003] Despite the remarkable achievements of FL in practice, its development still faces some challenges. As the application scope of federated learning continues to expand, the demand for model interpretability is also increasing. Interpretability plays a crucial role in ensuring model transparency and credibility, especially in scenarios where decisions have a significant impact and compliance with laws and regulations is required. Improving the interpretability of federated learning helps to understand the model decision-making process, enhance user trust in the system, and also facilitates the subsequent maintenance and adjustment of the model.

[0004] To improve the interpretability of distributed models, the Federated Fuzzy (Fuzzy Federated Learning, FFL) model has been studied. Local models can use gradient descent to optimize their parameters. The new federated fuzzy learning framework has achieved good results in network security. However, it should be noted that finding the global optimal solution and the best model is also one of the main challenges faced by the federated fuzzy problem. Many researchers have made many model attempts to address this problem in FFL, and many of these solutions have also conducted further research on personalized models for specific fields. However, existing FFL methods rarely can guarantee privacy protection while not relying on the setting of initial values and thus finding the global optimal solution most quickly. Although many studies have adopted methods such as modifying federation rules and using machine learning methods to assist FFL in dealing with abnormal initial values, their results often still require multiple attempts to obtain the best effect.

[0005] The existing federated fuzzy system technologies mainly have the following problems: 1. Traditional federated fuzzy systems are prone to falling into local optimal solutions when dealing with complex distributed data, resulting in a decline in the performance of the global model; 2. Optimization methods based on gradient descent have a slow convergence speed in the federated scenario and are vulnerable to data heterogeneity; 3. It is difficult to ensure model performance while protecting data privacy, especially when the client data distribution is uneven. Therefore, it is necessary to develop a new type of federated fuzzy system technology to improve the performance and generalization ability of the model in a distributed environment while ensuring privacy protection. Summary of the Invention

[0006] In view of the above deficiencies of the prior art, the present invention provides an optimization method for a federated fuzzy system based on particle information collaboration (Particle Information-based Collaborative Federated Fuzzy System, PICFFS). The present invention constructs a federated learning framework that takes into account privacy, interpretability, and generalization ability by integrating swarm intelligence optimization and fuzzy inference mechanisms.

[0007] The technical solution of the present invention is as follows:

[0008] An optimization method for a federated fuzzy system based on particle information collaboration, comprising the following steps:

[0009] The first step: Construct a federated learning environment, determine the number of clients N, the feature dimension n of the local dataset of each client, the number of membership functions m of the local fuzzy system, and the number of communication rounds T between the client and the server.

[0010] The second step: Client data processing:

[0011] 2.1 Establish a local fuzzy model based on ANFIS (Adaptive Neuro-Fuzzy Inference System) on each client, and adopt a Gaussian membership function:

[0012]

[0013] Among them, A(x) represents the membership degree of the input x to the fuzzy set. c is the center of the Gaussian function, and σ is the standard deviation of the Gaussian function, which controls the width of the curve. A larger sigma value makes the curve wider and flatter.

[0014] 2.2 Initialize the antecedent parameter Q a and the consequent parameter Q c : For a dataset with n input features and m fuzzy sets for each feature, relevant indicators of the model complexity can be directly calculated according to the network structure. Including the antecedent parameter Q a , the consequent parameter Q c .

[0015]

[0016] where and represent the center and standard deviation of the j-th membership function of the i-th feature. represents the coefficient related to the n-th input feature in the m-th n rule. m n represents the total number of combinations of fuzzy rules.

[0017] 2.3 Define each particle x in the particle swarm optimization algorithm i . In the PSO algorithm, each particle x i represents a complete set of parameters, consisting of Q a and Q c in two parts:

[0018]

[0019] where, x i represents the position vector of particle i, with a dimension of n·m + m n ·(n + 1). The position update and velocity update of each particle will act on all the parameters of Q a and Q c simultaneously to achieve multi-objective optimization. The optimization objective is to minimize the error between the output of the ANFIS model and the expected output by adjusting the antecedent and consequent parameters.

[0020] 2.4 Set the optimization objective function according to the task type. For classification problems, accuracy is used: In this article, for classification problems, the objective function is defined as:

[0021]

[0022] For regression problems, correlation is used:

[0023]

[0024] where y i is the true value. is the predicted value. is the mean of the true values. is the mean of the predicted values, and N represents the total number of samples.

[0025] 2.5 Run the particle swarm optimization algorithm on the local data for L iterations of training, update the particle positions and velocities, and obtain the optimal particle.

[0026] 2.6 Calculate the parameter change distance of the optimal particles before and after training: After L iterations, the positions of each particle will be continuously updated to obtain a better solution. The client sorts all particles according to the performance of the objective function and selects the current optimal particle as the analysis object. Specifically, for the optimal particle x best (i.e., the initial model with the best performance after substituting into the objective function) and the optimal particle x' best after training and optimization, their parameters will be different.

[0027] For a specific client I, since all particles have been sorted according to the performance of the objective function, the particle ranked first is the optimal particle. At this time, the antecedent parameter change distance of the first fuzzy set in the first feature of it can be defined as:

[0028]

[0029] Among them, represents the change distance of the first membership function parameter of the first feature of client I before and after training. By analogy, the complete particle distance Dis I of client I can be further defined, including the change amounts of all antecedent parameters and consequent parameters:

[0030]

[0031] Calculating this particle parameter distance can help measure the optimization amplitude and direction experienced by the client model before and after training, so as to provide a reference for the update of the global model and federated optimization.

[0032] 2.7 Upload the particle distance information and participation degree v I to the server: Assume that there are N clients in the system. When transmitting information between client I and the server, after client I completes this round of training, it needs to transmit the information and the particle distance information of client I in this round of training, and v I .

[0033]

[0034] Among them, v I represents the participation degree of client I in this federation. When v i is 0, it means that the client does not participate at all; when v i is 1, it means that the client reaches the highest participation degree.

[0035] Step 3: Server data processing:

[0036] 3.1 The server receives the particle distance information uploaded by all clients and performs weighted aggregation according to the client participation degree:

[0037]

[0038] 3.2 The server broadcasts the aggregated particle distance information to all clients.

[0039] Step 4: Client parameter update:

[0040] 4.1 The client receives the aggregated particle distance information broadcast by the server and updates the local model parameters:

[0041]

[0042] 4.2 Repeat steps 2 to 4 until the preset number of communication rounds T is reached.

[0043] Step 5: Final model output:

[0044] After completing T rounds of training, each client obtains the finally optimized local fuzzy system model, which can be used for local prediction tasks.

[0045] The advantages of the present invention include the following points:

[0046] 1) Break through the limitation that the traditional gradient descent optimization method is prone to falling into local optimal solutions, introduce the particle swarm optimization algorithm to globally search for the fuzzy system parameters, and improve the model performance.

[0047] 2) Design an aggregation strategy based on particle distance information, and only exchange the relative position features between particles instead of the original data or complete model parameters, significantly enhancing the privacy protection ability.

[0048] 3) Adopt a participation-weighted aggregation mechanism, which can adapt to heterogeneous data distribution environments and improve the robustness and generalization ability of the model in unbalanced data scenarios.

[0049] 4) Maintain the inherent interpretability advantage of the fuzzy system, improve the rule quality and discrimination by optimizing the rule parameters, and make the model decision-making process more transparent and reliable.

[0050] 5) Experiments prove that this method is superior to existing federated fuzzy methods on multiple classification and regression datasets, and has good convergence and parameter stability. Description of the Drawings

[0051] Figure 1 is the overall structure diagram of the PICFFS framework of the present invention.

[0052] Figure 2 is the interaction flow chart between the client and the server of the present invention.

[0053] Figure 3 is the schematic diagram of the calculation of particle distance information. Detailed implementation manners

[0054] The present invention will be described in detail below with reference to the accompanying drawings and embodiments:

[0055] In the first stage, the present invention combines a decentralized federated framework to distribute model updates and parameter optimization to each client. By dynamically adjusting the individual and global optimal solutions of the particle swarm, it realizes the efficient distributed optimization of model parameters in the federated learning process, ensuring data privacy while improving the convergence speed and accuracy of federated learning.

[0056] In the second stage, the present invention is based on the classical ANFIS fuzzy system and conducts adaptive optimization and evolutionary learning of fuzzy rules through the PSO algorithm. Each client generates and optimizes fuzzy rules according to local data, and then realizes the rule fusion of the global fuzzy system through the aggregation strategy from each client, ensuring the efficient removal of redundant rules in the model optimization process, thereby constructing an efficient and stable federated fuzzy system.

[0057] Table 1 Statistical information of the dataset

[0058]

[0059] Embodiment 1

[0060] An optimization method for a federated fuzzy system based on particle information collaboration, comprising the following steps:

[0061] The first step: Determine the number of clients for training and hyperparameters.

[0062] The second step: Build a local model on the client to learn the local private dataset, and then transfer the locally trained model data to the server.

[0063] The third step: Optimize the global model according to the local data obtained in the second step.

[0064] The fourth step: Distribute the optimized global model in the third step to the clients

[0065] The fifth step: Repeat the second to fourth steps repeatedly to obtain a stable and optimal model. Obtain the final classification result.

[0066] In Embodiment 1, the present invention uses 5 publicly available multi-view data and a synthetic dataset (MultiClassSim) for model construction and evaluation. The specific information of the real dataset is shown in Table 1.

[0067] Tables 2 and 3 respectively show the performance results of six methods on several datasets, covering regression and classification tasks. The best results in the tables are marked in bold, which can intuitively reflect the advantages and disadvantages of each method. Generally speaking, the performance of PICFFS is better than other methods on the vast majority of datasets, demonstrating its strong generalization ability and adaptability.

[0068] In the regression task, PICFFS performs excellently in the objective function Acc and can better fit the changing trend of continuous variables; in the classification task, its score in the objective function Corr also leads other methods, indicating its high classification accuracy. This benefits from the global search ability of the particle swarm optimization mechanism, which overcomes the defect that traditional methods are prone to falling into local optimal solutions, and at the same time effectively integrates the models of each node through the federated learning strategy.

[0069] These results prove the effectiveness of PICFFS, indicating that the combination of particle swarm optimization and fuzzy federated learning is a reliable method to improve the model performance.

[0070] Table 2 Comparison of algorithm performance based on classification datasets

[0071]

[0072]

[0073] Table 3 Comparison of algorithm performance based on regression datasets

[0074]

Claims

1. An optimization method for a federated fuzzy system based on particle information collaboration, characterized in that, It includes the following steps: The first step: Build a federated learning environment, determine the number of clients N, the feature dimension n of the local dataset of each client, the number of membership functions m of the local fuzzy system, and the number of communication rounds T between the clients and the server; The second step: Build a client module, establish a local fuzzy model based on the adaptive neuro-fuzzy inference system on each client, and perform local training through the particle swarm optimization algorithm; 2.1 Establish a local fuzzy model based on ANFIS on each client, and adopt the Gaussian membership function: Among them, A(x) represents the membership degree of the input x to the fuzzy set; c is the center of the Gaussian function, and σ is the standard deviation of the Gaussian function, which controls the width of the curve; 2.2 Initialize the antecedent parameter Q of the fuzzy system a and the consequent parameter Q c ; For a dataset with n input features, each feature having m fuzzy sets, including the antecedent parameters Q a , the consequent parameters Q c ; where and represent the center and standard deviation of the j-th membership function of the i-th feature; denotes the coefficient related to the n-th input feature in the m-th n rule; m n denotes the total number of combinations of fuzzy rules; 2.3 Define each particle x in the particle swarm optimization algorithm i , representing a complete set of parameters; 2.4 Set the optimization objective function according to the task type; 2.5 Run the particle swarm optimization algorithm on the local data for L iterations of training, update the particle positions and velocities, and obtain the optimal particle; 2.6 Calculate the parameter change distance of the optimal particle before and after training; 2.7 Upload the particle distance information and the participation degree v I to the server; The third step: The server module performs aggregation and broadcasting; 3.1 The server receives the particle distance information uploaded by all clients and performs weighted aggregation according to the client participation degree; 3.2 The server broadcasts the aggregated particle distance information to all clients; The update formula for 4.1 is as follows: 4.2 Repeat steps two to four until the preset number of communication rounds T is reached; The fifth step: After completing T rounds of training, each client obtains the finally optimized local fuzzy system model, which can be used for local prediction tasks.

2. The optimization method of a federated fuzzy system based on particle information collaboration according to claim 1, wherein, In step 2.3 described above, each particle x in the particle swarm optimization algorithm i represents a parameter set as follows: where \(x^i\) represents the position vector of particle \(i\), and its dimension is \(n\cdot m + m\) n ·(n + 1).

3. The optimization method of a federated fuzzy system based on particle information collaboration according to claim 1, characterized in that In step 2.4 described above, the optimization objective function for the classification problem is: The regression problem adopts correlation: where y*i is the true value, is the predicted value, is the mean of the true values, is the mean of the predicted values, and N represents the total number of samples.

4. The optimization method of a federated fuzzy system based on particle information cooperation according to claim 1, wherein, In step 2.6 described above, the specific operation is as follows: For the untrained optimal particle x best (i.e., the initial model that performs optimally after being substituted into the objective function) and the optimally trained particle x' best , the antecedent parameter change distance of the first fuzzy set in the first feature of Client I is: The complete particle distance Dis of Client I I is:

5. The optimization method of a federated fuzzy system based on particle information collaboration according to claim 1, characterized in that, In the third step described above, the method for the server to receive the particle distance information uploaded by all clients and perform weighted aggregation according to the client participation is as follows: where ν i represents the participation of the i-th client in this federation. When v i is 0, it means the client does not participate at all; when v i is 1, it means the client reaches the highest participation level.

Images