Federal learning training optimization method based on improved white shark intelligent optimization algorithm
By improving the White Shark intelligent optimization algorithm and dynamic communication control, the problems of high communication cost and high computational energy consumption in federated learning have been solved, achieving efficient training in heterogeneous device environments and improving model stability and resource utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-04-03
AI Technical Summary
Existing federated learning suffers from high communication costs, high computational energy consumption, and unstable model convergence performance in heterogeneous devices and resource-constrained environments. Existing optimization algorithms are prone to getting trapped in local optima in high-dimensional non-independent and identically distributed data environments, making it difficult to balance model accuracy and system resource consumption.
An improved White Shark intelligent optimization algorithm is introduced, combined with dynamic communication and parameter compression control mechanisms, to construct the Fed-IWSO federated learning system. The improved White Shark intelligent optimization algorithm is used to calculate parameter importance, perform screening and quantization compression, and use the benefit-cost ratio (RCR) for dynamic control to achieve a balance between communication efficiency and training performance.
It significantly reduces the amount of communication data, lowers communication overhead, achieves a balance between model accuracy and system energy consumption, improves training efficiency and stability, and is suitable for heterogeneous devices and complex network environments.
Smart Images
Figure CN121787461A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and distributed machine learning technology, specifically relating to a federated learning training optimization method based on an improved White Shark intelligent optimization algorithm, and particularly to an optimization technique for improving model training efficiency and communication performance in heterogeneous devices and resource-constrained environments. Background Technology
[0002] In today's data-driven era, massive amounts of data are widely distributed across personal devices and edge nodes. Traditional centralized machine learning methods require uploading data from devices to a central server for training. However, with the increasing prominence of data privacy and security issues, especially driven by data privacy protection laws such as the General Data Protection Regulation (GDPR), the privacy risks posed by centralized data storage have become a major social concern. To address this, Federated Learning (FL) has emerged. As a distributed machine learning method where data remains local, FL allows participating nodes to train models on local data and only upload model updates or parameter information, without transmitting the original data, thus achieving joint modeling while ensuring data privacy. Federated Learning has therefore gained widespread recognition for its effectiveness in privacy protection and data security, and its application prospects are particularly promising in data-sensitive fields such as mobile devices, the Internet of Things, healthcare, and finance.
[0003] Despite its privacy advantages, federated learning faces numerous challenges in practical applications. First, participants in federated learning systems are typically geographically distributed devices with varying computing power, storage resources, and network bandwidth, leading to significant differences in efficiency during training. In many federated learning applications, participating clients may be resource-constrained edge nodes such as smartphones and IoT devices, which often struggle to handle high-frequency data transmission and complex model computations. Furthermore, federated learning requires frequent communication between clients and servers for parameter synchronization and updates, generating substantial communication costs, especially in bandwidth-constrained or unstable network environments. Large data transmissions not only strain bandwidth but also introduce high latency. For applications requiring real-time responses, such as smart homes and real-time monitoring, increased communication latency severely impacts system performance and user experience. Therefore, reducing communication volume and improving communication efficiency while maintaining model accuracy is a key issue in federated learning research.
[0004] In addressing the challenges of communication and computational costs, recent research has primarily focused on optimizing communication efficiency and parameter selection. For example, methods such as gradient compression, parameter pruning, and sparsification have achieved some success by reducing the amount of data transmitted, thus lowering the communication burden. However, most of these methods focus on reducing the amount of data transmitted per transaction, failing to adequately consider the differences in resource constraints and data distribution among clients, making it difficult to flexibly adapt to the heterogeneity of real-world application scenarios. Especially in edge computing and IoT scenarios, the performance and bandwidth of client devices can vary significantly due to factors such as location and device type. Existing communication optimization methods often lack specificity, making it difficult to achieve efficient federated learning in real-world environments. Furthermore, most current communication optimization strategies primarily focus on improving model convergence speed and accuracy, while neglecting overall cost control and economic efficiency optimization during system operation. This lack of cost-benefit balance in design can lead to high resource consumption in practical deployments, thus limiting the promotion and application of federated learning.
[0005] To address these issues, the combination of intelligent optimization algorithms and federated learning has emerged as a new research direction. Intelligent optimization algorithms, such as Particle Swarm Optimization (PSO), Genetic Algorithm (GA), and Grey Wolf Optimization, possess strong global search capabilities and can improve the efficiency of federated learning in areas such as parameter selection and gradient compression. In some studies, intelligent optimization algorithms have been used to select the parameters most critical to model optimization transmitted on the client side, reducing transmission volume and lowering communication costs. However, while these studies have improved the efficiency of federated learning to some extent, their application scenarios are typically relatively limited, and most studies still prioritize model accuracy and convergence speed, lacking systematic optimization for cost-effectiveness. Furthermore, the application of intelligent optimization algorithms in federated learning also faces the practical constraint of limited device resources. When these algorithms are executed on edge devices, they often require high computational overhead, making it difficult to achieve a balance between cost and benefit.
[0006] As federated learning gains increasing attention, cost-effectiveness optimization is becoming a key factor hindering its widespread application. For cost-sensitive fields such as the Internet of Things (IoT) and smart homes, federated learning frameworks that only focus on model performance improvement are insufficient to meet practical needs. These scenarios require a federated learning framework capable of dynamically adjusting communication frequency and data transmission volume, while balancing communication, computation, and power costs to achieve better economic efficiency. However, most current federated learning frameworks lack comprehensive attention to this issue, limiting their promotion and implementation in commercial scenarios. Therefore, designing a federated learning framework that prioritizes profit maximization and integrates intelligent optimization algorithms, dynamic communication control, and cost monitoring has significant research and application value. Summary of the Invention
[0007] This invention aims to address the problems of high communication costs, high computational energy consumption, and unstable model convergence performance in existing federated learning training processes. Specifically, existing optimization algorithms are prone to getting trapped in local optima in high-dimensional non-independent identically distributed (Non-IID) data environments, failing to balance model accuracy and system resource consumption. This invention proposes a federated learning training optimization method that achieves a balance between communication efficiency and training performance in heterogeneous devices and resource-constrained scenarios by introducing an improved White Shark intelligent optimization algorithm and combining dynamic communication and parameter compression control mechanisms. To achieve the above objectives, this invention provides a federated learning training optimization method based on the improved White Shark intelligent optimization algorithm, which includes the following steps:
[0008] S1: Construct the Fed-IWSO federated learning system framework, which includes a client module, a server aggregation module, a dynamic control module, and a cost monitoring module. The client module is used to perform local model training, and the server aggregation module is used to aggregate client parameters and update the global model.
[0009] S2: After local model training is performed on the client, the importance coefficients of each model parameter are calculated using the improved White Shark intelligent optimization algorithm. Important parameters are screened and quantized and compressed, and only key parameters are uploaded to the server, thereby significantly reducing the amount of communication data.
[0010] The improved White Shark algorithm's individual position update formula is:
[0011]
[0012] in, Let X be the parameter vector for the i-th client. * X is the globally optimal parameter. rand For random reference individuals, β and γ are global and local search adjustment coefficients, respectively;
[0013] S3: After receiving the compressed parameters uploaded by the client, the server aggregation module performs dequantization and aggregation operations, and updates the globally optimal model parameters based on the improved White Shark intelligent optimization algorithm.
[0014] The client fitness function is defined as follows:
[0015] F i =w1×Acc i -w2×(C comm,i +C comp,i +C energy,i )
[0016] Among them, Acc i C represents the model accuracy. comp,i C comp,i C energy,iThese represent communication, computing, and energy costs, respectively, with w1 and w2 being weighting coefficients used to balance model performance and system cost.
[0017] S4: The cost monitoring module calculates the revenue-to-cost ratio (RCR) in real time. The calculation formula is as follows:
[0018]
[0019] Where E represents the model accuracy improvement rate, K is the profit coefficient, and C comm C comp C energy These are the costs of communication, computing, and energy consumption, respectively. The dynamic control module dynamically adjusts the communication frequency and parameter transmission ratio based on the cost-benefit ratio, achieving an adaptive balance between communication efficiency and economic benefits.
[0020] S5: To further improve the resource utilization efficiency during the training process, this invention designs a dynamic compression ratio update formula:
[0021] ρ t+1 =ρ t ×(1-λ×ΔRCR t )
[0022] Where, ρ t Let λ be the compression ratio in round t, λ be the learning rate coefficient, and ΔRCR be the compression ratio in round t. t This represents the change in the benefit-cost ratio. When benefits decrease, the system automatically increases the compression ratio to reduce communication costs, thereby achieving optimal overall efficiency.
[0023] S6: The system repeats the above training and optimization steps until the global model converges or reaches the preset accuracy threshold. This process enables adaptive collaborative optimization among multiple clients, improving the communication efficiency and energy utilization of federated learning.
[0024] The present invention has at least the following beneficial effects:
[0025] This invention introduces an improved White Shark intelligent optimization algorithm on the client side to achieve intelligent selection and quantization compression of model parameters, effectively reducing the amount of uploaded data and lowering the communication overhead of federated learning. The revenue-cost ratio (RCR) is introduced as a global control metric to dynamically control the communication frequency and compression ratio, achieving a balance between energy consumption and performance while ensuring model accuracy. Through chaotic initialization and adaptive weight update mechanisms, the global search capability and convergence accuracy of the improved White Shark intelligent optimization algorithm are enhanced, avoiding getting trapped in local optima and improving the stability of model training. This invention is applicable to various heterogeneous devices and complex network environments, and can be deployed in edge computing, IoT, and mobile smart terminals, exhibiting good scalability and versatility. Attached Figure Description
[0026] Figure 1 This is a system architecture diagram of the Fed-IWSO framework of the present invention, illustrating the interaction flow between the client, server, and dynamic control module.
[0027] Figure 2 This is a flowchart of the Fed-IWSO framework federated learning training optimization method of the present invention. Detailed Implementation
[0028] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that those skilled in the art can make various modifications and variations to the following embodiments without departing from the spirit and scope of the present invention, and all such modifications and variations fall within the protection scope of the present invention.
[0029] like Figure 1 As shown, this invention provides a federated learning training optimization method based on an improved white shark intelligent optimization algorithm. The system mainly includes the following functional modules:
[0030] Client Module: Used to perform local model training, calculate parameter importance coefficients, and quantize and compress important parameters before uploading.
[0031] The Server Aggregation Module receives parameters uploaded by the client, performs dequantization and aggregation operations, and updates the global model using the improved White Shark intelligent optimization algorithm.
[0032] Dynamic Control Module: Used to adjust the communication frequency and parameter upload ratio according to the benefit-cost ratio (RCR) to achieve a balance between system communication and performance.
[0033] Cost Monitoring Module: Used to calculate communication, calculation and energy costs in real time, and provide feedback information to the dynamic control module.
[0034] The modules are interconnected through communication links to form a federated training framework that is collaborative between the end, edge, and cloud.
[0035] like Figure 2 As shown, the method of the present invention includes the following steps:
[0036] S1: Before training begins, the server initializes the global model parameters W0 and distributes the model structure and hyperparameters to each client. Simultaneously, it initializes the population parameters of the improved white shark intelligent optimization algorithm, including the number of white sharks N, the maximum number of iterations T, the search coefficients β and γ, and the learning rate λ.
[0037] S2: Each client independently performs several rounds of gradient descent training on its local dataset to obtain a local model parameter set {W}. i Subsequently, the importance coefficients I of each parameter were calculated using the improved White Shark intelligent optimization algorithm. i This is done to filter key parameters and compress them.
[0038] The IWSO position update formula is defined as follows:
[0039]
[0040] in: Let X be the current parameter vector for the i-th client; * The globally optimal solution; X rand β and γ are random reference individuals; β and γ are dynamic weighting factors used to balance global search and local convergence.
[0041] By calculating the fitness function for each parameter:
[0042] F i =w1×Acc i -w2×(C comm,i +C comp,i +C energy,i )
[0043] The client is based on F i Upload parameters with high adaptability to achieve a balance between performance and energy consumption.
[0044] S3: The server aggregation module receives compressed parameter vectors from each client, performs dequantization and aggregation operations, and generates a new global model.
[0045]
[0046] Where, α i These are weighting coefficients calculated based on the client data size and fitness level.
[0047] The server-side utilizes an improved White Shark intelligent optimization algorithm to perform optimization search in the global parameter space and update the globally optimal solution W. * .
[0048] S4: The cost monitoring module calculates communication, computing, and energy costs, and evaluates system benefits using a benefit-cost ratio formula.
[0049]
[0050] Where: E represents the model accuracy improvement rate, K is the profit coefficient, and C... comm C comp C energyThese represent communication, computing, and energy costs, respectively. This metric is used to guide communication scheduling and compression rate adjustments.
[0051] S5: Based on the change in the benefit-cost ratio ΔRCR t The dynamic control module adjusts the compression ratio for the next round of communication.
[0052] ρ t+1 =ρ t ×(1-λ×ΔRCR t )
[0053] When revenue decreases, the compression ratio is automatically increased to reduce the amount of communication data, thereby minimizing overall costs. Furthermore, when the model convergence rate falls below a preset threshold, the dynamic control module will reduce the communication frequency to decrease system energy consumption and extend terminal lifespan.
[0054] S6: Repeat steps S2 to S5 until the model converges or reaches the set performance threshold. The server outputs the final global model parameters W. * Complete the federated learning optimization process.
[0055] In summary, this invention proposes a federated learning training optimization method based on an improved White Shark intelligent optimization algorithm. By jointly introducing adaptive parameter selection, dynamic compression ratio adjustment, and benefit-cost ratio feedback control mechanisms on both the client and server sides, a comprehensive optimization of communication efficiency, model accuracy, and system energy consumption is achieved. The system adaptively adjusts the parameter upload strategy and aggregation frequency by real-time monitoring of communication and computational load, combined with the multi-objective global search capability of the White Shark algorithm, thereby significantly improving training efficiency and model stability in heterogeneous devices and resource-constrained environments.
[0056] In further extended implementations, this invention can be combined with deep reinforcement learning models to automatically generate efficient task allocation and communication scheduling schemes based on task function characteristics (including computational resource requirements, model startup latency, etc.), server real-time status (CPU and memory load), and dynamic operating environment (network bandwidth, latency, and task backlog). The system achieves a dynamic balance between response time and energy consumption through multi-objective optimization strategies, thereby obtaining a federated learning scheduling strategy with optimal overall performance.
[0057] Although the present invention has been described in conjunction with specific embodiments, those skilled in the art will understand that various modifications, substitutions, or equivalent transformations can be made without departing from the principles and spirit of the invention, and all such modifications, substitutions, or transformations should fall within the protection scope of the present invention. The protection scope of the present invention is defined by the appended claims.
Claims
1. A federated learning training optimization method based on an improved white shark intelligent optimization algorithm, characterized in that, Includes the following steps: S1: Construct the Fed-IWSO federated learning framework, which includes a client module, a server aggregation module, a dynamic control module, and a cost monitoring module; S2: Local model training is performed in the client module. The importance coefficients of each model parameter are calculated using the improved White Shark intelligent optimization algorithm. Key parameters are selected, quantized and compressed, and then uploaded to the server aggregation module. S3: After receiving the compression parameters uploaded by the client, the server aggregation module performs dequantization and aggregation operations, and uses the improved White Shark intelligent optimization algorithm to optimize and update the global model; S4: The cost monitoring module calculates communication costs, computing costs, and energy consumption costs, and generates real-time cost monitoring indicators; S5: The dynamic control module adjusts the communication frequency and parameter transmission ratio based on the revenue-cost ratio (RCR) to achieve a balance between communication efficiency and economic benefits. S6: Repeat steps S2 to S5 until the global model converges or reaches the preset training accuracy.
2. The method according to claim 1, characterized in that, The improved white shark intelligent optimization algorithm includes the following steps: S1: Initialize the white shark population using chaotic mapping, setting the maximum number of iterations and the initial velocity; S2: Enhances early-stage global search capabilities and late-stage local search capabilities through a non-inertial weighted decreasing parameter optimization algorithm; S3: In each iteration, determine the global or local search mode based on the generated random number, and update the position and speed of the white shark; S4: Continuously update the fitness-optimal solution within the maximum number of iterations to obtain the globally optimal parameters.
3. The method according to claims 1 and 2, characterized in that, The individual position update formula of the improved white shark intelligent optimization algorithm is as follows: in: X represents the parameter vector of the i-th client in the t-th iteration; * X represents the globally optimal parameter; rand represents a random reference individual; β and γ are global and local search adjustment coefficients, respectively, used to balance global exploration and local convergence capabilities.
4. A federated learning training optimization method based on an improved white shark intelligent optimization algorithm, as described in claims 1 to 3, characterized in that, The fitness function F of the improved white shark intelligent optimization algorithm method i Defined as: F i =w1×Acc i -w2×(C comm,i +C comp,i +C energy,i ) Among them: Acc i C represents the model accuracy for client i; comm,i C comp,i C energy,i These represent communication cost, computing cost, and energy cost, respectively; w1 and w2 are weighting coefficients used to balance model performance and system energy consumption.
5. The method according to any one of claims 1 to 4, characterized in that, The formula for calculating the revenue-cost ratio (RCR) is: Where: E represents the model accuracy improvement rate; K is the revenue coefficient; C_comm, C_comp, and C_energy represent the communication, computation, and energy costs, respectively; the RCR value is used to guide the dynamic adjustment of communication frequency and parameter upload ratio.
6. The method according to any one of claims 1 to 5, characterized in that, The communication compression ratio is adaptively updated based on the change in the benefit-cost ratio using the following formula: r t+1 =ρ t ×(1-λ×ΔRCR t ) Where: ρ t ΔRCR represents the compression ratio of the t-th round of communication; λ represents the learning rate coefficient; ΔRCR t This represents the change in the benefit-cost ratio; using this formula, when benefits decrease, the compression ratio is automatically increased to reduce communication overhead, thereby achieving optimal cost-effectiveness.
7. The method according to any one of claims 1 to 6, characterized in that, The dynamic control module adaptively adjusts the communication frequency based on the model convergence rate and bandwidth utilization. When the convergence rate is lower than the preset threshold, the number of communication rounds is reduced to lower system energy consumption.
8. A federated learning training optimization system for implementing the method of any one of claims 1 to 7, characterized in that, include: (1) Client module, used to perform local model training and use the improved White Shark intelligent optimization algorithm to filter key parameters for compression and uploading; (2) Server aggregation module, used to receive parameters uploaded by the client, perform dequantization and aggregation operations and update the global model; (3) Dynamic control module, used to adjust the communication frequency and parameter upload ratio according to the revenue-cost ratio (RCR); (4) Cost monitoring module, used to calculate communication, calculation and energy consumption costs, and output optimization feedback signals.
9. The system according to claim 8, characterized in that, The dynamic control module and the cost monitoring module are connected through a feedback channel to update the communication control threshold in real time, thereby maximizing profits and optimizing the allocation of system resources.
10. A computer-readable storage medium having program instructions stored thereon that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.