Client and sample joint optimization method for federal learning of industrial Internet of Things

Abstract

The invention discloses a client and sample joint optimization method for federated learning of the industrial Internet of Things, which comprises the following steps: S1, constructing a cloud-side-end architecture, S2, initializing a cloud server, S3, executing importance perception data shuffling by terminal equipment, S4, screening clients participating in federated learning training by an edge server, and S4, carrying out data shuffling by the edge server. S7, the edge server executes synchronous aggregation and uploads the updated data to the cloud server; S8, the cloud server executes global aggregation and issues the updated data to the edge server for synchronization; S9, the steps S3-S8 are repeated, and S7, the selected client device executes grouping importance sampling; S7, local model training is executed and the updated data is uploaded to the edge server; and the model converges or reaches a preset training round. According to the method, the training efficiency, the convergence speed and the prediction precision can be remarkably improved in a resource constraint scene.

Description

Technical Field

[0001] This invention relates to the field of federated learning and industrial Internet of Things (IIoT) technology, and more specifically, to a federated learning optimization method applicable to resource-constrained IIoT scenarios. Background Technology

[0002] With the rapid development of the Industrial Internet of Things (IIoT), fixed equipment (such as surveillance cameras and temperature and humidity sensors) and mobile devices (such as inspection drones and logistics robots) in factories generate massive amounts of data daily. This data contains key information such as equipment operating status and production efficiency, serving as the core basis for intelligent industrial decision-making. However, this data is usually stored in a scattered manner on terminal devices and involves production privacy and trade secrets. Directly uploading it to the cloud for centralized machine learning training poses a risk of data leakage. Therefore, federated learning has become the core paradigm for privacy-preserving model training in IIoT scenarios. This paradigm achieves distributed collaborative training while ensuring data privacy through a process of "local training on terminal devices - uploading only model updates - cloud aggregation and optimization".

[0003] However, existing federated learning technologies face three key challenges when applied in industrial IoT scenarios, resulting in low training efficiency and poor model performance: a. Data heterogeneity issue: Due to differences in deployment scenarios and functional positioning, industrial IoT devices exhibit significant heterogeneity in data quantity, type, and distribution, violating the traditional machine learning assumption of "independent and identically distributed (i.i.d.)". Existing technologies often employ random client selection strategies, which easily select devices whose data distribution deviates from the global average, leading to a decrease in the training accuracy of the global model.

[0004] b. Device storage constraints: Most industrial IoT terminal devices have limited storage capacity and cannot support a large amount of local data; while existing federated learning methods generally assume that the client has sufficient storage capacity and do not consider the limitations of storage constraints on sample selection, which can easily lead to low-quality, redundant samples occupying storage space and affecting the effectiveness of training.

[0005] c. Dynamic streaming data adaptation problem: Industrial IoT device data is dynamically generated in streaming form, and the data distribution changes continuously over time. Traditional static data processing methods cannot update training samples in real time, making it difficult for the model to adapt to new data features and reducing its generalization ability.

[0006] Furthermore, existing federated learning methods often rely on random sampling for sample selection, which can easily lead to oversampling of low-importance samples during training iterations. These low-importance samples contribute minimally to model parameter updates, increasing redundant computational overhead and potentially causing model overfitting, further reducing training efficiency and final performance. In summary, existing technologies cannot meet the "efficient, accurate, and resource-adaptive" requirements of federated learning in Industrial Internet of Things (IIoT) scenarios, necessitating a targeted optimization solution.

[0007] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0008] The purpose of this invention is to address the above-mentioned problems by providing a joint optimization method for client and sample in federated learning for the Industrial Internet of Things (IIoT). This method effectively mitigates the performance degradation caused by data heterogeneity, reduces the computing power consumption of devices, and improves training efficiency, convergence speed, and prediction accuracy while ensuring device data privacy.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows: The client-sample joint optimization method for federated learning in the Industrial Internet of Things includes the following steps: S1. Construct a cloud-edge-device architecture consisting of cloud servers, edge servers, and IIoT terminal devices. S2. Initialize global model parameters on the cloud server. And calculate the true global data distribution. Complete system initialization and distribute global data. and global model parameters Distribute to each edge server, S3. The terminal device receives dynamic streaming data in real time and performs importance-aware data shuffling. This importance-aware data shuffling refers to determining whether to replace old data with new data based on the importance value of the samples and the device capacity. S4. Edge servers filter out data that meets the constraints and best approximates the distribution of aggregated data. The L devices serve as clients participating in the current round of federated learning training. S5. The selected client device performs group importance sampling on its local dataset to generate mini-batch data required for training. S6. The selected client device performs local model training and uploads the client's local model update data to the edge server. S7. The edge server performs synchronous aggregation and uploads the aggregated model update data to the cloud server. S8. The cloud server performs global aggregation, updates the global model, and distributes the updated global model parameters to the edge servers for synchronization. S9. Repeat steps S3 to S8 above until the model converges or reaches the predetermined number of training rounds.

[0010] Furthermore, step S3 includes the following steps: S31. Quantize the feedforward loss value based on the current global model, calculate the importance value of the newly incoming data, and simultaneously retrieve the importance values ​​of existing samples cached locally on the device. S32. Compare the importance values ​​of the new data and the cached samples. If the importance of the new data is higher than that of some samples in the cache, and the total amount of data after replacement does not exceed the device's storage capacity... If so, delete the low-importance samples from the cache, store the new data in the cache, and update the device's local dataset. And the column vector of data volume corresponding to the category labels of the newly stored data. ( (where F represents the number of categories in the data sample), and upload the data volume column vector. , to the edge server.

[0011] Furthermore, step S4 includes the following steps: S41. Column vector of the amount of data transmitted by each device within the group to each edge server. And generate the category data volume matrix of the group, as shown in the following formula:

[0012] S42. The edge server uses a 0-1 knapsack optimization model to calculate whether each device meets the selection criteria. The optimization objective function is: ,in The binary vector representing the device selection must satisfy the following constraint: the number of devices selected in each group is L, corresponding to the formula: ， Where K is the total number of devices in the group. It refers to the transpose of a vector with K rows and 1 column, where each element has a 1-column value.

[0013] Furthermore, step S5 includes the following steps: S51. The selected client device will use the local dataset. The sample is divided into d balanced and mutually exclusive groups, denoted as Sg. The number of samples in each group Sg is denoted as Sa and the number of samples is denoted as j. S52. Based on a polling mechanism, determine the sample groups whose importance needs to be updated in this round, and recalculate the importance values ​​of these samples. The remaining groups reuse historical importance results. S53. Calculate the selection probability of each sample group based on the exponential weighting function, and prioritize the sample group whose importance has been most recently updated, as shown in the following formula:

[0014] S54. Within the selected sample group, sample according to the proportion of sample importance, prioritizing samples with higher importance proportions, generating a sample size of... Small batches of data were used to complete sample screening.

[0015] This invention also provides a client-sample joint optimization system for federated learning in the Industrial Internet of Things (IIoT), comprising a cloud-edge-device architecture consisting of a cloud server, an edge server, and IIoT terminal devices, with federated learning modules deployed based on the cloud-edge-device architecture and information transmitted through a communication module; The system's cloud server is equipped with a global model, which is responsible for performing system initialization, global aggregation calculations, and transmitting global model parameters and the actual global data distribution to all edge servers through a communication module. The system's edge server is equipped with a training client selection module and an aggregation module. The training client selection module selects the aggregated data distribution that best approximates the true global data distribution. L devices serve as clients participating in this round of federated learning training; the aggregation module performs synchronous aggregation and uploads the aggregated model update data to the cloud server; The system's terminal device is equipped with a data shuffling module, an importance sampling module, and a local model training module. The data shuffling module determines whether to replace old data with new data based on the sample importance value and device capacity. The importance sampling module performs group importance sampling on the selected device to generate a small batch of data required for training. The local model training module performs client-side local model training and uploads the updated client-side local model data to the edge server.

[0016] Furthermore, during sampling, the importance sampling module updates the importance values ​​of samples based on a polling mechanism, prioritizes the sample group whose importance has been most recently updated, and then samples within the selected sample group according to the proportion of sample importance, generating a quantity of [number missing]. Small batches of data were collected to complete sample sampling.

[0017] In addition, the present invention also provides a client-sample joint optimization device for industrial IoT federated learning, comprising: a memory, a processor, and an industrial IoT federated learning client-sample joint optimization program stored on the memory and executable on the processor, wherein the industrial IoT federated learning client-sample joint optimization program is configured with a method for implementing industrial IoT federated learning client-sample joint optimization.

[0018] The present invention also provides a storage medium storing a client-sample joint optimization program for industrial IoT federated learning, wherein the client-sample joint optimization program for industrial IoT federated learning, when executed, implements the client-sample joint optimization method for industrial IoT federated learning.

[0019] By adopting the above technical solution, the present invention has the following beneficial technical effects: (1) On the basis of ensuring device data privacy, the layered cloud-edge-device architecture and client selection strategy effectively alleviate the model performance degradation caused by data heterogeneity, and make the training data distribution closer to the global real distribution.

[0020] (2) The value of samples is quantified by the group importance sampling strategy, and only high-importance samples are focused on training to avoid redundant calculations caused by oversampling of low-value samples and reduce the consumption of equipment computing power.

[0021] (3) Relying on the importance-aware data shuffling mechanism, the local cache samples are dynamically updated, which not only adapts to the dynamic streaming data characteristics of devices in the industrial Internet of Things scenario, but also strictly follows the device storage constraints to ensure the timeliness and high effectiveness of the cached data.

[0022] (4) It realizes the joint optimization of client selection and sample screening, adapts to the resource constraints of "limited storage, insufficient computing power and unstable communication" of the device, and significantly improves training efficiency, convergence speed and prediction accuracy compared with traditional federated learning methods. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the system of the present invention.

[0024] Figure 2 This is a flowchart of the method of the present invention.

[0025] Figure 3 This is a comparison of the convergence results between the present invention and existing technologies in non-independent and identically distributed systems. Detailed Implementation

[0026] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0027] Example 1: The client-sample joint optimization method of federated learning for industrial IoT of the present invention, such as... Figure 2 As shown, it includes the following steps: S1. Construct a cloud-edge-device architecture consisting of cloud servers, edge servers, and IIoT terminal devices. The cloud server layer is responsible for global model aggregation, cross-edge region coordination, and global data distribution analysis, maintaining the final global model. The edge server layer clusters industrial IoT devices into multiple groups based on geography or function, with each group corresponding to one edge server. Edge server nodes are responsible for client selection, local model aggregation, and device communication management within their group, while also connecting to the cloud server to upload and distribute model parameters. The device layer consists of various industrial IoT devices, such as surveillance cameras, mobile robots, smart sensors, and patrol drones, with data sets stored locally on each device.

[0028] Specifically, all IoT devices in the industrial park can be geographically clustered into groups, denoted as M, with devices within each group denoted as K, and numbered. Devices within the same group connect to the nearest edge server via a 5G network, and the edge server establishes a communication connection with the cloud server via the Internet, forming a "cloud-edge-device" data and model transmission link.

[0029] S2. Initialize global model parameters on the cloud server. Simultaneously, it receives basic information about each IoT device within each group from the edge server, including the local data distribution of the devices. ( ), and calculate the true global data distribution. The formula is as follows:

[0030] in Let k be the local data volume of device k in group m. For the data distribution of device k in group m, norm ( ) is the probability normalization function. Complete system initialization and distribute global data. and global model parameters Distribute to each edge server.

[0031] S3. The terminal device receives dynamic streaming data in real time and performs importance-aware data shuffling, specifically including the following steps: S31. Each IoT device collects streaming data in real time (such as production parameters monitored by sensors in real time, drone inspection data, etc.), and quantifies it based on the feedforward loss value of the current global model to calculate the importance value of newly incoming data. At the same time, it retrieves the importance values ​​of existing samples cached locally by the device. The formula for calculating the feedforward loss value of a single sample is as follows: , in The total number of categories, For the true label, if the sample belongs to the first... Class, then =1, otherwise 0 The model predicts that the sample belongs to the first... The probability value of a class. The higher the loss value of a single sample, the more important the sample is; high loss value samples represent data that the model has not yet fully fitted - these samples usually contain key features that the model has not captured. Prioritizing training on these samples can help the model quickly adjust parameters to cover the "uncaptured" features and accelerate convergence.

[0032] Samples with high loss values ​​contribute more to model gradient updates. During backpropagation, the gradient of the loss value directly determines the direction and magnitude of parameter updates: the higher the loss value, the larger the absolute value of its gradient, the stronger its influence on model parameter adjustment, and the more efficiently it can optimize global model performance.

[0033] S32. Compare the importance values ​​of the new data and the cached samples. If the importance of the new data is higher than that of some samples in the cache, and the total amount of data after replacement does not exceed the device's storage capacity... If so, delete the low-importance samples from the cache, store the new data in the cache, and update the device's local dataset. And the column vector of data volume corresponding to the category labels of the newly stored data. ( (F represents the number of categories in the data sample.) For privacy reasons related to federated learning, only the column vector of data volume will be uploaded subsequently. Instead of the original data, it is sent to the edge server for clients to choose from; if the new data is less important than all samples in the cache, the new data is discarded directly to ensure that only high-value samples are retained in the cache of each device.

[0034] S4. The edge server selects training clients, which includes the following steps: S41. Column vector of the amount of data transmitted by each device within the group to each edge server. And generate the category data volume matrix of the group, as shown in the following formula:

[0035] in This represents a column vector containing the data volume of device k in group m, where F is the number of sample category labels. Representative matrix For line F (Column), and simultaneously determine the number of client selections L in each iteration (e.g., L=10 devices per group); the edge server will , Using L as input parameters, the client selection process is initiated; S42. The edge server uses a 0-1 knapsack optimization model to calculate whether each device meets the selection criteria. The optimization objective function is: ,in The binary vector represents the device selection, where 1 indicates selected and 0 indicates unselected. It also satisfies the following constraint: the number of selected devices in each group is L, corresponding to the formula: ， Where K is the total number of devices in the group ( This refers to the transpose of a vector with K rows and 1 column, where each element is a 1-column vector.

[0036] Edge servers filter data to find those that meet the constraints and best approximate the distribution of aggregated data. The L devices serve as clients participating in this round of federated learning training.

[0037] S5. The selected client device performs group importance sampling on its local dataset to generate a mini-batch of data for training, which includes the following steps: S51. The selected client device will use the local dataset. The sample groups are divided into d balanced and mutually exclusive groups, denoted as Sg, to ensure that the computational burden is balanced among the groups. The number of samples in each sample group Sg is denoted as Sa, and the number of samples is denoted as j.

[0038] S52. Based on the polling mechanism, determine the sample groups whose importance needs to be updated in this round: if the first group was updated in the previous round, then the second group will be updated in this round, and so on. Only the importance value of the samples in this group is recalculated, while the historical importance results of the remaining groups are reused to reduce computational overhead.

[0039] The importance value of a sample is generally quantified based on the feedforward loss value of the current global model. The higher the loss value of a single sample, the higher its importance value.

[0040] S53. Calculate the selection probability of each sample group based on the exponential weighting function, and prioritize the sample group whose importance has been most recently updated, as shown in the following formula:

[0041] Let β be the number of iterations since the last update for sample group i, and let β be a negative scaling factor, such as β = -0.2.

[0042] S54. Within the selected sample group, sample according to the proportion of sample importance, prioritizing samples with higher importance proportions to generate the required sample size (assuming it is 1). For small-batch data, sample selection is completed, and the sample importance percentage is calculated using the following formula:

[0043] Let be the importance value of sample j in sample group i.

[0044] S6. The selected client device performs local model training and uploads the client's local model update data to the edge server. The specific steps are as follows: S61. The selected client device downloads the global model parameters from the previous round (t-1) from the edge server. Using mini-batch data as training data, the stochastic gradient descent algorithm is used for local training to obtain the local model parameters for the current round. .

[0045] S62. The client device will transfer local model parameters. and the scale of small batches of data on the client side Uploaded to the edge server of the group, without transmitting the original data, to ensure data privacy.

[0046] S7. The edge server performs synchronous aggregation and uploads the aggregated model update data to the cloud server.

[0047] The model update data includes model parameters. and the size of small batches of data within the group The aggregation formula is as follows: =

[0048] in, , The selected client device in group m during round t.

[0049] S8. The cloud server performs global aggregation, updates the global model, and distributes it to the edge server for synchronization. The specific steps are as follows: S81. The cloud server receives model parameters uploaded by edge servers from all groups. The updated global model parameters are obtained by performing global aggregation calculations based on the weights of the small batch data sizes between groups. The formula is as follows: =

[0050] in, .

[0051] S82. The cloud server will store global model parameters. The data is distributed to all edge servers, which then synchronize it to the IoT devices in their respective groups.

[0052] S9. Repeat steps S3 to S8 above until the model converges or reaches the predetermined number of training rounds.

[0053] We evaluate the method used in this invention based on the FEMNIST training dataset to verify its accuracy loss despite a significant reduction in computational overhead. The FEMNIST training dataset exhibits significant non-independent and identically distributed characteristics, effectively simulating the data heterogeneity challenges in the Industrial Internet of Things (IIoT). The experimental settings are as follows: total number of devices K=350, number of groups M=10, number of devices selected within the factory per round L=10, and learning rate η=0.01. Figure 3 As shown, experimental results demonstrate that the proposed joint optimization method (Ours) significantly outperforms the traditional FedAvg and FedProx algorithms in both convergence speed and final model accuracy, and closely approximates the theoretical performance upper limit (Full-IS) for full sample importance calculation. Specifically, in the first 100 training rounds, thanks to the precise capture of high-loss samples by the group importance sampling mechanism, the model can skip redundant data and directly focus on key features, resulting in an extremely high upward slope and achieving 80% accuracy in approximately 80 rounds. Ultimately, the proposed method converges to a high accuracy of approximately 92.5%, closely approximating the theoretical performance upper limit for full sample importance calculation, strongly validating the advantages of the method employed in this invention: since sample importance evolves smoothly with training iterations, by polling the grouped samples within the client in a grouped manner, only the sample importance of client 1 / d is updated, achieving significant computational savings with minimal computational cost. This allows the federated learning model to maintain excellent prediction accuracy and robustness even in resource-constrained industrial IoT environments.

[0054] Example 2: Additionally, this invention also provides a client-sample joint optimization system for federated learning in the Industrial Internet of Things (IIoT), such as... Figure 1As shown, the system comprises a cloud-edge-device architecture consisting of cloud servers, edge servers, and IIoT terminal devices, and a federated learning module is deployed based on this architecture. The cloud server layer is responsible for global model aggregation, cross-edge region coordination, and global data distribution analysis, maintaining the final global model. The edge server layer clusters industrial IoT devices into multiple groups based on geography or function, with each group corresponding to one edge server. Edge server nodes are responsible for client selection, local model aggregation, and device communication management within their group, while also connecting to the cloud server to upload and distribute model parameters. The device layer consists of various industrial IoT devices, such as surveillance cameras, mobile robots, smart sensors, and patrol drones, with data sets stored locally on each device.

[0055] The system also includes a communication module, through which cloud servers, edge servers, and IIoT terminal devices in the system transmit information.

[0056] The system's terminal device is equipped with a data shuffling module. This module determines whether to replace old data with new data based on sample importance values ​​and device capacity. Specifically, if the sample importance value of newly incoming data is higher than that of old data in the device's cache, and the total data volume after replacing the old data in the cache with new data does not exceed the device's storage capacity, then the old data cached in the device is replaced with new data.

[0057] The system's edge server is equipped with a training client selection module, which selects clients based on an objective function. Choose the aggregated data distribution that best approximates the actual global data distribution. The L devices serve as clients participating in this round of federated learning training.

[0058] The system's terminal device is also equipped with an importance sampling module. On the selected device, this module updates the importance value of a group of samples based on a polling mechanism, prioritizing the sample group whose importance has been most recently updated. Then, within the selected sample group, it samples according to the proportion of sample importance, generating a sample size of... Small batches of data were collected to complete sample sampling.

[0059] The system's terminal device is also equipped with a local model training module, which uses the global model parameters from the previous round. Using a small batch of data as training data, local training is performed to obtain the local model parameters for the current round. Then, local model parameters and the scale of small batches of data on the client side Uploaded to the edge server of the group via the communication module.

[0060] The system's edge server is equipped with an aggregation module, which is based on the quantity of small batches of data from devices within the group. Weights on client-side local model parameters Perform aggregation calculations to obtain the aggregated model parameters. Then model parameters and the size of this small batch of data. Uploaded to the cloud server via the communication module.

[0061] The system's cloud server is equipped with a global model, which is responsible for performing system initialization, including initializing global model parameters. and the true global data distribution The global model then transmits this data to the edge server via a communication module; it is also responsible for determining the scale of small-batch data between groups. Weights on model parameters Perform global aggregation calculations to obtain the updated global model parameters. Then the global model parameters The data is transmitted to all edge servers via the communication module.

[0062] Furthermore, this embodiment is merely a basic description of the client and sample joint optimization system for federated learning in the industrial Internet of Things of the present invention. For technical details not described in detail in this embodiment, please refer to the methods provided in any embodiment of the present invention, which will not be repeated here.

[0063] Example 3: Those skilled in the art will clearly understand that the systems and methods of the above embodiments can be implemented using software plus necessary general-purpose hardware platforms. Of course, they can also be implemented using hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, node packaging device, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0064] Therefore, the present invention also provides a client-sample joint optimization device for industrial IoT federated learning, comprising: a memory, a processor, and an industrial IoT federated learning client-sample joint optimization program stored in the memory and executable on the processor, wherein the industrial IoT federated learning client-sample joint optimization program is configured with a method for implementing industrial IoT federated learning client-sample joint optimization.

[0065] In addition, the present invention also provides a storage medium storing a client and sample joint optimization program for industrial Internet of Things federated learning.

[0066] In reality, when deploying equipment or programs, a program may execute all steps, or it may execute only one step, with multiple programs working together to achieve the full process. Therefore, when the client and sample joint optimization program of the Industrial Internet of Things federated learning is executed, it implements all or some of the processes in the client and sample joint optimization method of Industrial Internet of Things federated learning.

[0067] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification, or direct or indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A client-sample joint optimization method for federated learning in the Industrial Internet of Things, characterized in that, Includes the following steps: S1. Construct a cloud-edge-device architecture consisting of cloud servers, edge servers, and IIoT terminal devices. S2. Initialize global model parameters on the cloud server. And calculate the true global data distribution. Complete system initialization and distribute global data. and global model parameters Distribute to each edge server, S3. The terminal device receives dynamic streaming data in real time and performs importance-aware data shuffling. This importance-aware data shuffling refers to determining whether to replace old data with new data based on the importance value of the samples and the device capacity. S4. Edge servers filter out data that meets the constraints and best approximates the distribution of aggregated data. The L devices serve as clients participating in the current round of federated learning training. S5. The selected client device performs group importance sampling on its local dataset to generate mini-batch data required for training. S6. The selected client device performs local model training and uploads the client's local model update data to the edge server. S7. The edge server performs synchronous aggregation and uploads the aggregated model update data to the cloud server. S8. The cloud server performs global aggregation, updates the global model, and distributes the updated global model parameters to the edge servers for synchronization. S9. Repeat steps S3-S8 until the model converges or reaches the predetermined number of training rounds.

2. The client-sample joint optimization method for federated learning in the Industrial Internet of Things as described in claim 1, characterized in that, Step S3 includes the following steps: S31. Quantize the feedforward loss value based on the current global model, calculate the importance value of the newly incoming data, and simultaneously retrieve the importance values ​​of existing samples cached locally on the device. S32. Compare the importance values ​​of the new data and the cached samples. If the importance of the new data is higher than that of some samples in the cache, and the total amount of data after replacement does not exceed the device's storage capacity... If the data is low importance, delete the low importance samples from the cache, store the new data in the cache, and update the data volume column vector of the device's local dataset and the corresponding category labels of the newly stored data. and upload the data volume column vector. , to the edge server.

3. The client-sample joint optimization method for federated learning in the Industrial Internet of Things as described in claim 1, characterized in that, Step S4 includes the following steps: S41. Column vector of the amount of data transmitted by each device within the group to each edge server. And generate the category data volume matrix of the group, as shown in the following formula: S42. The edge server uses a 0-1 knapsack optimization model to calculate whether each device meets the selection criteria. The optimization objective function is: ,in The binary vector representing the device selection must satisfy the following constraint: the number of devices selected in each group is L, corresponding to the formula: ， Where K is the total number of devices in the group. It refers to the transpose of a vector with K rows and 1 column, where each element has a 1-column value.

4. The client-sample joint optimization method for federated learning in the Industrial Internet of Things as described in claim 1, characterized in that, Step S5 includes the following steps: S51. The selected client device will use the local dataset. The sample is divided into d balanced and mutually exclusive groups, denoted as Sg. S52. Based on a polling mechanism, determine the sample groups whose importance needs to be updated in this round, and recalculate the importance values ​​of these samples. The remaining groups reuse historical importance results. S53. Calculate the selection probability of each sample group based on the exponential weighting function, and prioritize the sample group whose importance has been most recently updated, as shown in the following formula: in Let be the number of iterations since the last update for sample group i, and β be the negative scaling factor. S54. Within the selected sample group, sample according to the proportion of sample importance, prioritizing samples with higher importance proportions, generating a sample size of... Small batches of data were used to complete sample screening.

5. A client-sample joint optimization system for federated learning in the Industrial Internet of Things, characterized in that, It includes a cloud-edge-device architecture consisting of cloud servers, edge servers, and IIoT terminal devices. Federated learning modules are deployed based on the cloud-edge-device architecture, and information is transmitted through communication modules. The system's cloud server is equipped with a global model, which is responsible for performing system initialization, global aggregation calculations, and transmitting global model parameters and the actual global data distribution to all edge servers through a communication module. The system's edge server is equipped with a training client selection module and an aggregation module. The training client selection module selects the aggregated data distribution that best approximates the true global data distribution. L devices serve as clients participating in this round of federated learning training; the aggregation module performs synchronous aggregation and uploads the aggregated model update data to the cloud server; The system's terminal device is equipped with a data shuffling module, an importance sampling module, and a local model training module. The data shuffling module determines whether to replace old data with new data based on the sample importance value and device capacity. The importance sampling module performs group importance sampling on the selected device to generate a small batch of data required for training. The local model training module performs local model training on the client and uploads the updated local model data to the edge server.

6. The client-sample joint optimization system for federated learning in the Industrial Internet of Things as described in claim 5, characterized in that, The importance sampling module updates the importance values ​​of samples based on a polling mechanism during sampling, prioritizing the sample group whose importance has been most recently updated. Then, within the selected sample group, sampling is performed according to the sample importance percentage, generating a sample size of [number missing]. Small batches of data were collected to complete sample sampling.

7. A device for joint optimization of clients and samples in federated learning for the Industrial Internet of Things, characterized in that, include: The system includes a memory, a processor, and an industrial IoT federated learning client and sample joint optimization program stored on the memory and capable of running on the processor. The industrial IoT federated learning client and sample joint optimization program is configured to implement the industrial IoT federated learning client and sample joint optimization method as described in claim 1.

8. A storage medium, characterized in that, The storage medium stores a client and sample joint optimization program for industrial IoT federated learning. When executed, the client and sample joint optimization program for industrial IoT federated learning implements the client and sample joint optimization method for industrial IoT federated learning as described in claim 1.

Images