Heterogeneous data federal learning dynamic batch processing training method facing edge device

Abstract

The invention relates to the technical field of federated learning and edge computing, in particular to an edge-device-oriented heterogeneous data federated learning dynamic batch processing training method. An edge device registers and reports data to a server, the server constructs a device portrait library and issues initial parameters, and during training, the edge device calculates a resource availability score, generates a dynamic batch processing strategy, determines an adaptive Batch Size, triggers gradient accumulation when resources are insufficient, dynamically resamples Non-IID data, adjusts a learning rate by using an adaptive optimizer, and carries out dynamic batch processing on the Non-IID data. And the server aggregates the model update quantity to obtain a global model, determines whether to terminate training according to the loss change of the verification set, and if not, calculates a comprehensive convergence index to adjust a global learning rate, and after parameters are updated, the edge device continues to train. According to the method, a training strategy can be dynamically adjusted according to edge device resources and data characteristics, and the training efficiency and model performance of federated learning in an edge device heterogeneous data scene are improved.

Description

Technical Field

[0001] The present invention relates to the field of federated learning and edge computing technology, and in particular to a dynamic batch training method for heterogeneous data federated learning for edge devices. Background Art

[0002] With the vigorous development of the Internet of Things and edge computing, edge devices are becoming increasingly important in the field of data collection and processing. However, edge devices have significant differences in computing power, storage capacity, and network bandwidth, resulting in highly heterogeneous data and computing resources between devices. In this case, how to efficiently process heterogeneous data from different devices and sensors and make full use of device resources has become a key challenge facing edge computing.

[0003] Although federated learning technology has achieved data training in a distributed environment, the integration of heterogeneous data and the differences between devices are still difficult problems that need to be solved urgently. This problem seriously affects the overall training efficiency and progress, and hinders the effective training of the model. Therefore, how to effectively solve the data and device heterogeneity problems while improving the efficiency of model training on edge devices is a pressing issue. Based on this, a dynamic batch training method for heterogeneous data federated learning on edge devices is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide a dynamic batch training method for heterogeneous data federated learning on edge devices to solve the problems raised in the above background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The dynamic batch training method for heterogeneous data federated learning on edge devices includes the following steps:

[0007] Step 1: The edge device registers with the server and reports static and real-time data. The server then builds an edge device profile library to store the static and real-time data of the edge device. At the same time, the server sends the initial weight coefficient, initial global learning rate, and maximum / minimum batch size.

[0008] Step 2: Based on the data stored in the edge device profile library, the edge device generates a dynamic batch processing strategy. Before each round of training, it calculates a resource availability score to reflect its own resource sufficiency. The adaptive batch size is calculated through the dynamic batch processing function. If resources are tight, gradient accumulation is triggered.

[0009] Step 3: The edge device dynamically resamples the Non-IID data, which refers to data that is uneven and inconsistent across different edge devices, and uses an adaptive optimizer to dynamically adjust the edge device learning rate based on the adaptive batch size to control the model parameter update step size.

[0010] Step 4: After the edge device completes local training, the model is updated with the change amount The metadata including adaptive BatchSize, data volume, and resource availability score is uploaded to the server. The server uses the aggregation weight calculation formula and weighted average formula to derive the global model.

[0011] Step 5: The server sets a threshold ϵ using statistical analysis based on historical training data. The server then determines the loss change of the global model on the validation set, which serves as an indicator of the performance fluctuation of the model on the validation set. If the loss change is lower than the threshold ϵ, training is terminated and the optimal global model is generated. Otherwise, the server calculates a comprehensive convergence index on the validation set to evaluate the convergence status of the global model. The server then flexibly adjusts the global learning rate to control the global model parameter update step size. The server also sends the updated maximum batch size and weight coefficient to the edge device, which then regenerates the dynamic batch processing strategy.

[0012] Preferably, the method for the server to construct a device portrait library includes:

[0013] The server establishes a unified data reception specification, clarifying the format and content requirements of the data reported by edge devices. Memory capacity is measured in GB, the maximum allowable batch size is an integer, CPU models follow specific naming conventions, local data volume is counted by sample number, category distribution is expressed as the percentage of samples in each category, and data update frequency is expressed in time intervals. The remaining battery power is expressed as a percentage, and CPU utilization and memory utilization are decimals.

[0014] The server performs comprehensive data verification on the reported data. For data that fails verification, it promptly sends an error message to the device, requesting re-reporting or correction. At the same time, it removes duplicate data, outliers, and noise data. Based on the different characteristics of static data and real-time data, appropriate database storage is selected. For static data, the relational database MySQL is selected to create a static capability table. The edge device ID is used as the primary key, and auxiliary indexes are established based on the local data volume range. The associated storage memory capacity, maximum allowable batch size, CPU model, local data volume, category distribution, and data update frequency are stored. For real-time data, the time series database InfluxDB is used to create a dynamic status table. The edge device ID is also used as the identifier. A secondary index is established based on the time field. The remaining power, CPU usage, and memory usage are dynamically updated and stored in chronological order. The edge device sends heartbeat packets to the server at a set period. The heartbeat packets carry the current real-time data. The device ID is used to establish a relationship between the two databases, enabling the acquisition of complete profile information for the same device from different databases.

[0015] Preferably, the method for obtaining the adaptive batch size includes:

[0016] The adaptive batch size is obtained according to the dynamic batch processing function, wherein the dynamic batch processing function includes:

[0017] in, For the Adaptive Batch Size for edge devices, and is the preset weight coefficient, and , is the basic Batch Size value, and They are the minimum and maximum Batch Size, For the The amount of local data on each edge device, For the Resource availability score of each edge device, The function is a clipping function, is the maximum amount of local data in all edge devices, Score the maximum resource availability among all edge devices;

[0018] described The function limits the input value to a specified range.

[0019] The calculation formula is:

[0020]

[0021] The calculation formula for the resource availability score is: ;

[0022] in, is the percentage of remaining power, and its weight is , is the ratio of available memory to total memory, and its weight is , is the CPU idle rate, and its weight is .

[0023] Preferably, the method for triggering gradient accumulation includes:

[0024] In federated learning, edge devices have different computing resources and memory capacities. When edge device resources are insufficient, directly training in the conventional way may lead to excessive computing burden or even training interruption. Therefore, it is necessary to trigger gradient accumulation judgment.

[0025] The edge device will calculate the resource availability score With pre-set resource availability score threshold For comparison, if , it indicates that the current device resources are tight, triggering the gradient accumulation mechanism. If Then follow the normal training process and use adaptive Batch Size for training.

[0026] Preferably, the method for the edge device to perform dynamic resampling on Non-IID data includes:

[0027] During local training, the edge device performs local training data at regular intervals.

[0028] Perform comprehensive statistics, compare the category distribution of local data with the category distribution of global data, use KL divergence to measure the difference in category distribution, determine the resampling target, select the random oversampling method, and for minority class samples, randomly copy their samples until the desired number of samples is reached. Replace the original local training data with the new data obtained after resampling, and record relevant information about the resampling, including the resampled category, the resampling method used, and the number of samples before and after resampling. The calculation formula of the KL divergence is:

[0029] ;

[0030] Among them, P is the category distribution probability of local data, and Q is the category distribution probability of global data;

[0031] The larger the KL divergence value, the more significant the difference in category distribution between local data and global data, and the smaller the value, the more similar the distribution of the two is.

[0032] Preferably, the adaptive optimizer dynamically adjusts the learning rate of the edge device according to the adaptive batch size:

[0033] The edge device learning rate is dynamically adjusted using a linear scaling rule, which is: ,

[0034] in is the basic Batch Size value, For the Adaptive Batch Size for edge devices, For the edge device learning rate, is the global learning rate;

[0035] The linear scaling rule adjusts the learning rate accordingly based on the change in batch size. The principle is that when the batch size increases, the variance of the gradient estimate decreases. At this time, the learning rate can be appropriately increased to speed up the convergence. When the batch size decreases, the variance of the gradient estimate increases, and the learning rate needs to be reduced to ensure the stability of training.

[0036] Preferably, the method for obtaining the global model includes:

[0037] Receive model update changes and metadata uploaded by each device, including adaptive batch size, data volume, and resource score. Calculate the global model using the aggregation weight formula and weighted average formula.

[0038] The aggregation weight calculation formula and weighted average formula are:

[0039] Aggregation weight calculation formula:

[0040]

[0041] in For the Aggregate weight of edge devices, For the Adaptive BatchSize for edge devices, For the Resource availability score of each edge device, For the The amount of data locally held by each edge device;

[0042] Weighted average calculation formula:

[0043] in, For the Aggregate weight of edge devices, Update the delta for the model, is the global model parameter of the t-th round of training;

[0044] The model update change The calculation formula is: ;

[0045] in, are the local model parameters obtained after t rounds of local training.

[0046] Preferably, the method by which the server determines the loss change of the global model on the validation set includes:

[0047] The server extracts 10%-20% of the data from all edge devices participating in federated learning to form a validation set. The server collects the model update changes and metadata uploaded by each device. The server uses the current global model to perform forward propagation on the validation set, calculates the loss value between the model prediction result and the true label of the validation set, records the loss value by round, and forms a loss value sequence. The server calculates the average loss change over multiple consecutive rounds. The average loss change calculation formula is:

[0048] Among them, the average is the average loss change, is the loss value of the global model on the validation set after the tth round of training, is the loss value of the global model on the validation set after the tKth round of training, The window size of the number of training epochs for computing the average loss change;

[0049] Set the loss change threshold ϵ, if the calculated average If the global model converges, the server terminates the training and outputs the current global model as the optimal model. If it is greater than the threshold ϵ, the server continues training.

[0050] Preferably, the method of elastically adjusting the global learning rate according to the obtained comprehensive convergence index includes:

[0051] The comprehensive convergence index is obtained according to the comprehensive convergence index formula, which is: ;

[0052] Among them, DCI is the comprehensive convergence index, is the weight of the loss change rate, is the weight of the accuracy stability metric and , is the loss change rate, is a measure of accuracy stability;

[0053] The loss change rate formula is: ;

[0054] Among them, K is the number of selected time window rounds, is the loss value of the global model on the validation set after the tth round of training, is the loss value of the global model on the validation set after the tKth round of training;

[0055] The calculation formula of the accuracy stability metric is: ;

[0056] in, For the After rounds of training, the average accuracy of the global model on the validation set is, For the Round of training to The mean of the average accuracy of the global model of the round on the validation set;

[0057] Threshold and is the threshold of the comprehensive convergence index, which varies according to the specific federated learning scenario. Usually, the low threshold is between 0.05-0.1, and the high threshold is between 0.2-0.3. Less than the low threshold When , it can be multiplied by a decay factor less than 1 to reduce the global learning rate. Greater than high threshold When , it can be multiplied by a growth factor greater than 1 to increase the global learning rate. exist and When , the current global learning rate remains unchanged.

[0058] Preferably, the method for obtaining the updated maximum batch size and weight coefficient includes:

[0059] How to obtain the updated maximum batch size: The server adjusts the maximum batch size based on the performance of the global model on the validation set. If the global model converges quickly and the accuracy stability continues to improve under the maximum batch size, the maximum batch size can be increased. If the model's accuracy stability on the validation set decreases while the accuracy stability on the training set increases, or convergence is slow, the maximum batch size can be reduced.

[0060] Method for obtaining updated weight coefficients: The server evaluates the contribution of each edge device to the global model update based on the aggregation weight calculation formula and the model update changes and metadata uploaded by each edge device. For edge devices with large contributions, their corresponding weight coefficients are increased when calculating the aggregation weight. , for edge devices with small contribution, reduce , while re-adjusting To ensure , the specific adjustment can refer to the formula:

[0061]

[0062] The contribution of device i can be measured by comparing the accuracy improvement of the global model on the validation set before and after the update of device i. The average contribution is the average of the contributions of all devices, and the maximum contribution is the maximum value of the contributions of all devices.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] A dynamic batch training method for heterogeneous data federated learning on edge devices significantly improves training efficiency, model performance, and adaptability by addressing the heterogeneous computing resources and data of edge devices. This method introduces a dynamic batching mechanism that automatically adjusts the batch size based on the device's computing resource availability score and local data volume, ensuring improved training efficiency when resources are sufficient and guaranteed training stability when resources are tight. This avoids training interruptions due to insufficient resources and fully utilizes the resource advantages of edge devices.

[0065] When device resources are limited, the gradient accumulation mechanism is triggered to reduce computing and memory usage, allowing resource-constrained edge devices to be trained smoothly, improving the adaptability and robustness of federated learning and ensuring the continuity of the training process. At the same time, to meet the challenges of Non-IID data, the device regularly calculates the category distribution differences between local and global data, uses KL divergence to measure the deviation of data distribution, and oversamples minority class samples. This strategy effectively balances data distribution, improves the model's adaptability to different types of data, and enhances the model's generalization and stability.

[0066] In addition, based on adaptive Batch Size, a linear scaling rule is used to dynamically adjust the learning rate of edge devices. This allows the learning rate to flexibly respond to changes in Batch Size, optimizes the convergence process, and improves the model convergence speed and training efficiency. The server elastically adjusts the global learning rate through a comprehensive convergence index to ensure that the appropriate learning rate can be used in different training stages, effectively avoiding oscillations and accelerating the convergence process, further improving the training effect of the global model.

[0067] Finally, the server adopts a weighted averaging strategy to optimize the aggregation process of the global model by combining the model update amount and related metadata uploaded by the device, ensuring that the global model can integrate the advantages of each device and accurately reflect the overall data characteristics, thereby improving the accuracy and generalization ability of the model. At the same time, by monitoring the changes in the validation set loss and the convergence index, the server can accurately judge the convergence status of the model, ensuring that the training process is not affected by overtraining, and optimizing the model quality and practicality.

[0068] Overall, this method breaks through the bottleneck of traditional federated learning on edge devices, improves resource utilization, optimizes model training effects, and provides strong support for realizing efficient and stable edge computing applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 A flowchart of the steps of the dynamic batch training method for heterogeneous data federated learning for edge devices provided by the present invention; DETAILED DESCRIPTION

[0071] To make the objectives, technical solutions, and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other implementations obtained by those of ordinary skill in the art without inventive effort are within the scope of protection of the present invention.

[0072] Examples, such as Figure 1 As shown in FIG, the dynamic batch training method for heterogeneous data federated learning for edge devices includes the following steps:

[0073] Step 1: The edge device registers with the server and reports static and real-time data. The server then builds an edge device profile library to store the static and real-time data of the edge device. At the same time, the server sends the initial weight coefficient, initial global learning rate, and maximum / minimum batch size.

[0074] Step 2: Based on the data stored in the edge device profile library, the edge device generates a dynamic batch processing strategy. Before each round of training, it calculates a resource availability score to reflect its own resource sufficiency. The adaptive batch size is calculated through the dynamic batch processing function. If resources are tight, gradient accumulation is triggered.

[0075] Step 3: The edge device dynamically resamples the Non-IID data, which refers to data that is uneven and inconsistent across different edge devices, and uses an adaptive optimizer to dynamically adjust the edge device learning rate based on the adaptive batch size to control the model parameter update step size.

[0076] Step 4: After the edge device completes local training, the model is updated with the change amount The metadata including adaptive BatchSize, data volume, and resource availability score is uploaded to the server. The server uses the aggregation weight calculation formula and weighted average formula to derive the global model.

[0077] Step 5: The server sets a threshold ϵ using statistical analysis based on historical training data. The server then determines the loss change of the global model on the validation set, which serves as an indicator of the performance fluctuation of the model on the validation set. If the loss change is lower than the threshold ϵ, training is terminated and the optimal global model is generated. Otherwise, the server calculates a comprehensive convergence index on the validation set to evaluate the convergence status of the global model. The server then flexibly adjusts the global learning rate to control the global model parameter update step size. The server also sends the updated maximum batch size and weight coefficient to the edge device, which then regenerates the dynamic batch processing strategy.

[0078] Furthermore, the working principle of the present invention is described below by way of examples:

[0079] Assume that there are 100 edge devices participating in federated learning to train an image classification model to identify three animals: cats, dogs, and rabbits. These edge devices include mobile phones, smart cameras, etc., which have different computing power, storage capacity, and network bandwidth. The amount and category distribution of local data are also different.

[0080] When an edge device first connects to the server, it reports static data, such as the phone's 4GB memory capacity, maximum batch size of 64, Snapdragon 865 CPU model, 1000 local data images (300 cat, 500 dog, and 200 rabbit images, respectively), and a daily data update frequency. Simultaneously, it periodically reports real-time data via heartbeat packets, such as the current remaining battery life of 70%, CPU utilization of 30%, and memory utilization of 40%. After receiving the data, the server verifies it according to unified data reception specifications and inserts the static data into the static capability table of the relational database MySQL and the real-time data into the dynamic state table of the time series database InfluxDB. Using the device ID as the primary key, it creates auxiliary indexes based on the time field and the local data volume range to build a device profile. The server then issues initial weight coefficients (e.g., all set to 0.1), initial global learning rate (set to 0.001), and maximum and minimum batch sizes (maximum set to 128, minimum set to 16).

[0081] Before each round of training, each edge device calculates the resource availability score. For example, a smart camera device has 80% remaining power, 60% available memory, and 70% CPU idle rate, and the preset weights = 0.3, = 0.3 and =0.4, then its resource availability score = 0.3×80% + 0.3×60% + 0.4×70% = 70%. Then, the device calculates the adaptive batch size based on the dynamic batch function. Assuming =0.6, =0.4, the basic Batch Size value is 32, the local data volume of the device is 800, the maximum local data volume of all devices is 1500, the maximum resource availability score is 90%, the minimum and maximum Batch Sizes are 16 and 128 respectively, and the adaptive Batch Size is calculated = , approximately 43. If the device resource availability score is lower than the preset threshold (such as 60%), the gradient accumulation mechanism is triggered; otherwise, normal training is performed using the adaptive batch size.

[0082] During local training, the edge device collects statistics on the local training data every 5 rounds. Taking a mobile phone device as an example, the proportion of cat, dog, and rabbit images in its local data is 30%, 50%, and 20% respectively, while the proportion of these three types of images in the global data is 40%, 40%, and 20% respectively. By calculating the KL divergence to measure the difference in category distribution, it is determined that the cat sample is a minority class sample. The random oversampling method is selected, and the number of cat samples is expected to reach 400, that is, cat samples are randomly copied until the number requirement is met, and then the original training data is replaced with the resampled data, and the resampling information is recorded. At the same time, the device uses a linear scaling rule to adjust the learning rate. If the basic Batch Size value is 32, the adaptive Batch Size is 43, and the global learning rate is 0.001, then the device learning rate = .

[0083] After completing local training, the edge device uploads the model update changes and metadata (adaptive batch size, data volume, and resource score) to the server. After receiving the data, the server calculates the aggregate weight of each device using the aggregation weight calculation formula. For example, if a device has an adaptive batch size of 50, a resource availability score of 75%, and a local data volume of 1200, its aggregation weight is calculated according to the aggregation weight calculation formula. Then, the weighted average formula is used to aggregate the model update changes of each device to obtain the global model.

[0084] The server extracts 15% of the data from all edge devices to form a validation set. The server uses the current global model to perform forward propagation on the validation set, calculates the loss value between the predicted result and the true label, and records it. Assuming that the loss values ​​of 5 consecutive rounds (K=5) are 0.5, 0.45, 0.42, 0.4, and 0.38 respectively, the average loss change is calculated according to the average loss change formula. If the loss change threshold ϵ=0.03 is set, since -0.024 <0.03, the global model is determined to have converged, the training is terminated, and the current global model is output as the optimal model. If it has not converged, the comprehensive convergence index is calculated, such as the loss change rate weight =0.6, accuracy stability measurement weight = 0.4. After calculation according to the corresponding formula, if the comprehensive convergence index is less than the lower threshold, the global learning rate is reduced; if it is greater than the upper threshold, the global learning rate is increased; if it is within the threshold range, it remains unchanged. At the same time, the server adjusts the maximum batch size according to the performance of the global model on the validation set, adjusts the weight coefficient according to the contribution of each device to the global model update, and then sends the updated parameters to the edge device. The edge device regenerates the dynamic batch processing strategy and starts a new round of training.

[0085] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A dynamic batch training method for heterogeneous data federated learning on edge devices, characterized by: The method comprises the following steps: Step 1: The edge device registers with the server and reports static and real-time data. The server then builds an edge device profile library to store the static and real-time data of the edge device. At the same time, the server sends the initial weight coefficient, initial global learning rate, and maximum / minimum batch size. Step 2: Based on the data stored in the edge device profile library, the edge device generates a dynamic batch processing strategy. Before each round of training, it calculates a resource availability score to reflect its own resource sufficiency. The adaptive batch size is calculated through the dynamic batch processing function. If resources are tight, gradient accumulation is triggered. Step 3: The edge device dynamically resamples the Non-IID data, which refers to data that is uneven and inconsistent across different edge devices, and uses an adaptive optimizer to dynamically adjust the edge device learning rate based on the adaptive batch size to control the model parameter update step size. Step 4: After the edge device completes local training, the model is updated with the change amount The metadata including the adaptive batch size, data volume, and resource availability score is uploaded to the server. The server uses the aggregation weight calculation formula and weighted average formula to derive the global model. Step 5: The server sets a threshold ϵ using statistical analysis based on historical training data. The server then determines the loss change of the global model on the validation set, which serves as an indicator of the performance fluctuation of the model on the validation set. If the loss change is lower than the threshold ϵ, training is terminated and the optimal global model is generated. Otherwise, the server calculates a comprehensive convergence index on the validation set to evaluate the convergence status of the global model. The server then flexibly adjusts the global learning rate to control the global model parameter update step size. The server also sends the updated maximum batch size and weight coefficient to the edge device, which then regenerates the dynamic batch processing strategy.

2. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 1 is characterized in that: The method for the server to construct a device portrait library includes: When an edge device is registered for the first time, it carries the edge device ID and reports static data and real-time data to the server through periodic heartbeat packets. The static data is inserted into the static capability table of the relational database MySQL, and the real-time data is inserted into the dynamic status table of the time series database InfluxDB. The server uses the edge device ID as the primary key, establishes auxiliary indexes based on the time field and local data volume range, and builds an edge device portrait library. The static data includes memory capacity, maximum allowed batch size, CPU model, local data volume, category distribution, and data update frequency. The real-time data includes remaining power, CPU usage, and memory usage.

3. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 2, characterized in that: The method for obtaining the adaptive batch size includes: The adaptive batch size is obtained according to the dynamic batch processing function, wherein the dynamic batch processing function includes: , in, For the Adaptive Batch Size for edge devices, and is the preset weight coefficient, and , is the basic Batch Size value, and They are the minimum and maximum Batch Size, For the The amount of local data on each edge device, For the Resource availability score of each edge device, The function is a clipping function, is the maximum amount of local data in all edge devices, Score the maximum resource availability among all edge devices; The calculation formula for the resource availability score is: ; in, is the percentage of remaining power, and its weight is , is the ratio of available memory to total memory, and its weight is , is the CPU idle rate, and its weight is .

4. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 3 is characterized in that: The method for triggering gradient accumulation includes: The gradient accumulation refers to a strategy to ensure the stability and continuity of training when the edge device resources are tight. The edge device calculates the resource availability score With pre-set resource availability score threshold For comparison, if , it indicates that the current device resources are tight, triggering the gradient accumulation mechanism. If Then follow the normal training process and use adaptive Batch Size for training.

5. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 4 is characterized in that: The method for the edge device to perform dynamic resampling on Non-IID data includes: During the local training process, the edge device conducts comprehensive statistics on the local training data at regular intervals, compares the category distribution of local data with the category distribution of global data, uses KL divergence to measure the category distribution difference, determines the resampling target, and selects a random oversampling method. For minority class samples, its samples are randomly copied until the desired number of samples is reached. The new data obtained after resampling replaces the original local training data, and relevant information about the resampling is recorded, including the resampled category, the resampling method used, and the number of samples before and after resampling.

6. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 5, characterized in that: The adaptive optimizer dynamically adjusts the edge device learning rate according to the adaptive batch size: The edge device learning rate is dynamically adjusted using a linear scaling rule, which is: , in is the basic Batch Size value, For the Adaptive Batch Size for edge devices, For the edge device learning rate, is the global learning rate.

7. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 6, characterized in that: The method for obtaining the global model includes: Receive model update changes uploaded by each device The metadata includes adaptive batch size, data volume, and resource availability score. The global model is derived through the aggregation weight calculation formula and weighted average formula. The aggregation weight calculation formula and weighted average formula are: Aggregation weight calculation formula: ， in For the Aggregate weight of edge devices, For the Adaptive Batch Size for edge devices, For the Resource availability score of each edge device, For the The amount of data locally held by each edge device; Weighted average calculation formula: ; in, For the Aggregate weight of edge devices, Update the delta for the model, are the global model parameters for the tth round of training.

8. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 7, characterized in that: The method for the server to determine the loss change of the global model on the validation set includes: The server extracts 10%-20% of the data from all edge devices participating in federated learning to form a validation set. The server collects the model update changes and metadata uploaded by each device. The server uses the current global model to perform forward propagation on the validation set, calculates the loss value between the model prediction result and the true label of the validation set, records the loss value by round, and forms a loss value sequence. The server calculates the average loss change over multiple consecutive rounds. The average loss change calculation formula is: , Among them, the average is the average loss change, is the loss value of the global model on the validation set after the tth round of training, is the loss value of the global model on the validation set after the tKth round of training, The window size of the number of training epochs for computing the average loss change; Set the loss change threshold ϵ, if the calculated average If the global model converges, the server terminates the training and outputs the current global model as the optimal model. If it is greater than the threshold ϵ, the server continues training.

9. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 8, characterized in that: The method for elastically adjusting the global learning rate according to the obtained comprehensive convergence index includes: The comprehensive convergence index is obtained according to the comprehensive convergence index formula, which is: ; Among them, DCI is the comprehensive convergence index, is the weight of the loss change rate, is the weight of the accuracy stability metric and , is the loss change rate, is a measure of accuracy stability; The loss change rate formula is: ; Among them, K is the number of selected time window rounds, is the loss value of the global model on the validation set after the tth round of training, is the loss value of the global model on the validation set after the tKth round of training; The calculation formula of the accuracy stability metric is: ; in, For the After rounds of training, the average accuracy of the global model on the validation set is, For the Round of training to The mean of the average accuracy of the global model of the round on the validation set; when Less than the low threshold When , it can be multiplied by a decay factor less than 1 to reduce the global learning rate. Greater than high threshold When , it can be multiplied by a growth factor greater than 1 to increase the global learning rate. exist and When between, keep the current global learning rate unchanged, the threshold and It is the threshold of the comprehensive convergence index, which varies according to the specific federated learning scenario. Usually, the low threshold is between 0.05-0.1, and the high threshold is between 0.2-0.

3.

10. The method for dynamic batch training of heterogeneous data federated learning for edge devices according to claim 9, characterized in that: The method for obtaining the updated maximum batch size and weight coefficient includes: The server adjusts the maximum batch size based on the performance of the global model on the validation set. If the global model converges quickly and the accuracy stability continues to improve under the maximum batch size, the maximum batch size can be increased. If the model's accuracy stability on the validation set decreases while the accuracy stability on the training set increases, or convergence is slow, the maximum batch size can be reduced. The server calculates the aggregate weight based on the model update change uploaded by each edge device. and metadata, evaluate the contribution of each edge device to the global model update, and increase the corresponding weight coefficient of the edge device with large contribution when calculating the aggregation weight , for edge devices with small contribution, reduce , while re-adjusting To ensure .

Images