Federal learning model collaborative training and privacy protection system for industrial big data

Abstract

The application relates to the technical field of federal learning and industrial data security, and discloses a federal learning model cooperative training and privacy protection system for industrial big data, which comprises an industrial data access layer, a local federal training layer, a federal cooperative scheduling layer, a privacy protection reinforcement layer, a model aggregation verification layer and a model deployment application layer, each layer is cooperatively linked from top to bottom, the privacy security and efficient cooperative training of the industrial big data are realized, the application has strong adaptability, can effectively adapt to the characteristics of the industrial big data, such as time sequence, heterogeneity and massiveness, a time sequence weight factor is introduced through an improved federal learning algorithm, personalized training hyperparameter configuration is combined, the model training precision and training efficiency are improved, and the problem of poor adaptability of the existing system is solved, the application has high privacy protection strength, high cooperative efficiency, reliable aggregation results, and strong practicality and expandability.

Description

Technical Field

[0001] This invention belongs to the field of federated learning and industrial data security technology, specifically a federated learning model collaborative training and privacy protection system for industrial big data. Background Technology

[0002] With the rapid development of the Industrial Internet, massive amounts of heterogeneous data have accumulated in industrial scenarios, including equipment operation data, production process data, and environmental monitoring data. This industrial big data is the core foundation for achieving intelligent manufacturing, process optimization, and fault prediction. However, industrial big data often involves core corporate secrets and sensitive information. Significant data barriers exist between different industrial enterprises and different production nodes, posing serious privacy risks during data aggregation and sharing, and hindering the efficient utilization of data.

[0003] Federated learning, a distributed machine learning technique where "data is available but not visible," effectively avoids the aggregation of raw data by training models locally at each data holder and only uploading model parameters for aggregation, thus providing a feasible path for the collaborative utilization of industrial big data. However, existing federated learning systems still have many shortcomings in the application of industrial big data scenarios: First, it has poor adaptability. Industrial big data is characterized by time sequence, heterogeneity, and massive volume. Existing federated learning algorithms have not been optimized for the characteristics of industrial data, resulting in low training efficiency and model accuracy that is difficult to meet the needs of industrial scenarios. Second, the privacy protection is insufficient. Existing systems mostly use a single encryption method and lack full-process privacy protection for the training process, parameter transmission, and data storage. They are vulnerable to attacks such as parameter tampering and data theft, and cannot resist collusive attacks between nodes and servers. Third, the collaborative scheduling capability is weak, and it is unable to dynamically allocate training tasks according to the computing resources and network status of each local node. This can easily lead to problems such as unbalanced load and inconsistent training progress. Furthermore, it is difficult to recover quickly after training is interrupted, which affects training efficiency. Fourth, the aggregation results are unverifiable. The existing system cannot ensure that the global model parameters after aggregation have not been tampered with, nor can it verify the legality of the parameter source, which poses a risk of unreliable aggregation results. Fifth, relying on trusted third parties to generate keys increases system deployment costs and security risks, and makes it difficult to adapt to the distributed deployment needs of industrial scenarios.

[0004] To address the aforementioned technical challenges, there is an urgent need to design a federated learning system that is adapted to the characteristics of industrial big data, possesses efficient collaborative training capabilities and strong privacy protection capabilities, and whose aggregation results are verifiable and do not rely on trusted third parties. This system would enable the privacy and security of industrial big data, as well as its efficient collaborative utilization, thereby promoting the development of intelligent manufacturing. Summary of the Invention

[0005] In response to the above situation and to overcome the shortcomings of existing technologies, this invention provides a federated learning model collaborative training and privacy protection system for industrial big data, which effectively solves the problems raised in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a federated learning model collaborative training and privacy protection system for industrial big data, comprising an industrial data access layer, a local federated training layer, a federated collaborative scheduling layer, a privacy protection enhancement layer, a model aggregation and verification layer, and a model deployment and application layer. Each layer works in synergy from top to bottom to achieve privacy security and efficient collaborative training of industrial big data. The industrial data access layer is used to complete the distributed access and standardized preprocessing of multi-source heterogeneous industrial data. The local federated training layer is used to complete personalized model training based on preprocessed local industrial data. The federated collaborative scheduling layer is used to realize the collaborative control of each local training node and the aggregation node; The privacy protection enhancement layer is used to provide full-process privacy protection for data, parameters and communication processes during training. The model aggregation and verification layer is used to perform secure aggregation and authenticity verification of the model parameters uploaded by each local training node; The model deployment application layer is used to deploy the aggregated global model to industrial scenarios and realize application feedback.

[0007] Preferably, the industrial data access layer includes a multi-source heterogeneous data access module, a data cleaning and standardization module, a data fragmentation and desensitization module, and a data quality verification module; The multi-source heterogeneous data access module adopts a multi-protocol adaptation architecture, supports distributed access of industrial sensors, PLCs, SCADA systems, MES systems and industrial databases, and is compatible with multiple industrial communication protocols such as Modbus, OPC UA, MQTT and HTTP, enabling real-time acquisition of time-series monitoring data, equipment operation data, production process data and environmental parameter data. The data cleaning and standardization module is used to remove outliers, fill in missing values, unify data formats, and normalize dimensions of the collected industrial big data to generate standardized industrial datasets. The data segmentation and desensitization module is used to divide the standardized industrial dataset into training segments and validation segments according to preset segmentation rules. Sensitive fields in the training segments are preprocessed using a combination of format desensitization and value desensitization. The sensitive fields include core equipment parameters, production formula data, and confidential process parameters. The data quality verification module is used to verify the integrity, consistency, and accuracy of the preprocessed industrial data, discard unqualified data, and generate local training data that meets the requirements of federated training.

[0008] Preferably, the local federated training layer is deployed on each industrial participant node, and each industrial participant node corresponds to a local federated training unit. The local federated training unit includes a model initialization module, a personalized training configuration module, a local training execution module, a training parameter extraction module, and a parameter preprocessing module. The model initialization module is used to receive global initial model parameters issued by the federated collaborative scheduling layer and complete the initialization configuration of the local model. The global initial model parameters are preset based on the needs of industrial scenarios and are adapted to the time-series and high-dimensional characteristics of industrial big data. The personalized training configuration module is used to configure personalized training hyperparameters according to the type, scale and training objectives of local industrial data. The hyperparameters include learning rate, number of iterations, batch size and regularization coefficient, and support dynamic adjustment during the training process. The local training execution module adopts an improved federated learning algorithm to complete the iterative training of the local model based on local training data. The improved federated learning algorithm is based on the FedAvg algorithm optimization and introduces the time series weight factor of industrial data to improve the training adaptability of time series industrial data. The training parameter extraction module is used to extract the parameter update amount of the local model after each round of local training. The parameter update amount includes the weight parameter increment and the bias parameter increment. The parameter preprocessing module is used to normalize and sparsify the extracted parameter update amounts, remove invalid parameter update amounts, and reduce data transmission volume and privacy leakage risks.

[0009] Preferably, the federated collaborative scheduling layer includes a collaborative control center, a node status monitoring module, a training task allocation module, a communication link management module, and a training progress synchronization module; The collaborative control center, as the core scheduling unit of the system, is used to receive feedback information from modules at each layer, issue collaborative scheduling instructions, and coordinate the collaborative work of each module. The node status monitoring module is used to monitor the computing resource utilization, network bandwidth, training progress and node online status of each local federated training node in real time, and generate a node status evaluation report. The computing resource utilization includes CPU utilization, GPU utilization and memory utilization. The training task allocation module is used to allocate global training tasks to each local federated training node in a reasonable manner according to the node status evaluation report and training task requirements, using a dynamic task allocation algorithm to achieve load balancing. The dynamic task allocation algorithm completes the allocation based on node performance score and task weight. The node performance score is determined by a combination of computing resources, network status and historical training efficiency. The communication link management module is used to build encrypted communication links, select the optimal communication path, monitor the stability of the communication links, and automatically switch to the backup communication link when the link is abnormal to ensure the continuity of data transmission. The training progress synchronization module is used to synchronize the training progress of each local federated training node with the collaborative control center. It records the number of iterations, parameter upload time and training accuracy for each round of training. When the training progress of a certain node lags behind, it issues a progress adjustment instruction to ensure that the global training progress is consistent.

[0010] Preferably, the privacy protection enhancement layer includes a multi-layer encryption module, a differential privacy enhancement module, an access control module, and an attack detection module; The multi-layer encryption module adopts a three-level encryption architecture of "transmission encryption - storage encryption - computation encryption". Transmission encryption uses the TLS1.3 protocol to encrypt the parameter transmission process, storage encryption uses the AES-256 algorithm to encrypt and store local training data and model parameters, and computation encryption uses the homomorphic encryption algorithm to encrypt the intermediate calculation results during the training process, so as to realize model training in ciphertext state. The differential privacy enhancement module is used to add adaptive noise to the parameter update after the local model parameters are extracted. The intensity of the adaptive noise is dynamically adjusted according to the local data scale and privacy protection level, so as to ensure privacy and security while avoiding excessive noise from affecting the model accuracy. The access control module adopts a role-based access control (RBAC) mechanism, which divides access permissions into different levels, clarifies the access scope of each role to data, models and parameters, and only users with corresponding permissions can perform relevant operations. At the same time, all access operation logs are recorded for subsequent auditing. The attack detection module is used to monitor abnormal behaviors during the training process in real time, including parameter tampering attacks, malicious node attacks, and data theft attacks. It uses anomaly detection algorithms to identify attack behaviors. When an attack is detected, it immediately blocks the attack, isolates malicious nodes, and sends alarm information to the collaborative control center.

[0011] Preferably, the model aggregation and verification layer includes a parameter security aggregation module, an aggregation result verification module, a model accuracy evaluation module, and a model optimization and adjustment module; The parameter security aggregation module is used to receive the preprocessed parameter update amount uploaded by each local federated training node, and to complete the aggregation calculation of global model parameters using a weighted aggregation algorithm. The weights of the weighted aggregation algorithm are determined comprehensively based on the training data scale, data quality and training accuracy of each local node, so as to realize the optimization and update of the global model. The aggregation result verification module is used to verify the authenticity and integrity of the aggregated global model parameters using a bilinear aggregation signature algorithm, ensuring that the aggregation result has not been tampered with and that the parameters come from legitimate local training nodes. After the verification is passed, the global model parameters are sent to each local training node. If the verification fails, the aggregation process is retried. The model accuracy evaluation module is used to perform accuracy testing on the aggregated global model using preset accuracy evaluation indicators. The accuracy evaluation indicators include accuracy, recall, F1 score and time series prediction error. When the model accuracy does not reach the preset threshold, the retraining process is triggered. The model optimization and adjustment module is used to dynamically adjust the global training hyperparameters and aggregate weights based on the accuracy evaluation results and feedback information during the training process, thereby optimizing the training algorithm and improving the adaptability and accuracy of the global model.

[0012] Preferably, the model deployment application layer includes a model conversion module, a deployment adaptation module, a real-time inference module, and an application feedback module; The model conversion module is used to convert the aggregated global model into a deployment format adapted to industrial scenarios, supporting multi-terminal deployment on edge computing devices, industrial servers and cloud platforms; The deployment and adaptation module is used to personalize the deployed model according to the hardware configuration and application requirements of different industrial scenarios, adjust the model inference speed and resource utilization, and achieve seamless connection between the model and industrial equipment and systems. The real-time inference module is used to receive real-time data from industrial scenarios, perform real-time inference through the deployed global model, and output inference results, including equipment failure prediction, production quality prediction, and process parameter optimization suggestions. The application feedback module is used to collect inference results, accuracy performance and changes in industrial scenario requirements during the model application process, generate application feedback reports, and feed them back to the federated collaborative scheduling layer and the model aggregation and verification layer to provide data support for the iterative optimization of the model.

[0013] Preferably, the local training execution module further includes a training interruption recovery unit, which is used to save the current training state and parameter information when the training is interrupted due to abnormal conditions such as power failure or network interruption of the local training node. When the abnormality is recovered, the training can continue to be completed based on the saved training state without restarting the training process, thereby improving training efficiency.

[0014] Preferably, the collaborative control center also includes a key management unit for generating, distributing, updating, and destroying encryption keys required by each module of the system. It adopts a secret sharing algorithm to achieve distributed key management, avoiding the risk of privacy leakage caused by a single node holding all keys. The key update cycle can be dynamically adjusted according to the privacy protection level.

[0015] Preferably, the differential privacy enhancement module employs a dynamic privacy budget allocation algorithm, which dynamically allocates the privacy budget based on the number of iterations in each training round, data sensitivity, and model accuracy requirements, achieving an optimal balance between privacy protection and model accuracy. The privacy budget allocation process is encrypted to ensure that the allocation strategy is not leaked.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention has strong adaptability and can effectively adapt to the characteristics of industrial big data, such as time sequence, heterogeneity, and massive volume. By introducing time sequence weight factors through an improved federated learning algorithm and combining personalized training hyperparameter configuration, it improves the model training accuracy and efficiency, and solves the problem of poor adaptability of existing systems. 2. This invention offers high privacy protection by employing a three-level encryption architecture of "transmission encryption - storage encryption - computation encryption". Combined with differential privacy enhancement, access control and attack detection technologies, it achieves full-process privacy protection during training, data storage, parameter transmission and computation, resisting various attacks such as parameter tampering, data theft and collusion attacks. It does not rely on trusted third parties to generate keys, reducing security risks and deployment costs, and solving the problem of insufficient privacy protection in existing systems. 3. This invention has high collaborative efficiency. It achieves load balancing through a dynamic task allocation algorithm, and combines training interruption recovery, communication link redundancy backup, and training progress synchronization mechanisms to avoid efficiency decline caused by training interruption, load imbalance, and communication anomalies, thereby improving the overall collaborative training efficiency and solving the problem of low collaborative efficiency in existing systems. 4. The aggregation results of this invention are reliable. It introduces a bilinear aggregation signature algorithm to verify the authenticity and integrity of the global model parameters after aggregation, ensuring that the aggregation results have not been tampered with and that the parameters come from legitimate nodes, thus solving the problem that the aggregation results of existing systems are not verifiable. 5. This invention is highly practical and scalable, supports multi-terminal deployment (edge ​​computing devices, industrial servers, cloud platforms), and can be adapted to different industrial scenarios (equipment fault prediction, production quality prediction, process parameter optimization). Through the application feedback mechanism, the model can be continuously iterated and optimized, and it can be widely used in fields such as intelligent manufacturing and industrial internet, promoting the privacy, security and efficient utilization of industrial big data. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0018] In the attached diagram: Figure 1 This is a flowchart of the collaborative training process of the present invention; Figure 2 This is a block diagram of the privacy protection logic of the present invention; Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] Depend on Figures 1-2 This invention relates to a federated learning model collaborative training and privacy protection system for industrial big data, comprising an industrial data access layer, a local federated training layer, a federated collaborative scheduling layer, a privacy protection enhancement layer, a model aggregation and verification layer, and a model deployment and application layer. These layers work together from top to bottom to form a complete industrial big data federated learning collaborative training and privacy protection system. The specific composition and function of each module are as follows: 1. Industrial Data Access Layer The industrial data access layer is the system's data input point, used to complete the distributed access and standardized preprocessing of multi-source heterogeneous industrial data, providing high-quality, standardized training data for subsequent federated training. It includes a multi-source heterogeneous data access module, a data cleaning and standardization module, a data fragmentation and de-identification module, and a data quality verification module. The specific functions of each module are as follows: 1) Multi-source heterogeneous data access module: Adopting a multi-protocol adaptation architecture, it integrates multiple industrial communication protocols such as Modbus, OPC UA, MQTT and HTTP, enabling distributed access of industrial sensors, PLC (Programmable Logic Controller), SCADA (Supervisory Control and Data Acquisition), MES (Manufacturing Execution System) and industrial databases. It can collect time-series monitoring data, equipment operation data, production process data and environmental parameter data in industrial scenarios in real time. It supports both batch acquisition and real-time acquisition modes. The batch acquisition cycle can be set to 1-60 minutes, and the real-time acquisition latency does not exceed 100ms, ensuring the timeliness and comprehensiveness of data acquisition.

[0021] 2) Data Cleaning and Standardization Module: Addressing issues such as outliers, missing values, inconsistent formats, and inconsistent dimensions in industrial big data, this module uses preset cleaning rules to process the collected data. Outlier removal employs the 3σ principle, missing value completion uses interpolation based on time-series trends, the data format is standardized to JSON, and dimension normalization uses the min-max normalization algorithm to map the data to the [0,1] interval, generating a standardized industrial dataset to ensure data consistency and usability.

[0022] 3) Data Sharding and Anonymization Module: The standardized industrial dataset is divided into training shards and validation shards according to preset sharding rules. The sharding ratio can be set to 7:3, 8:2, or 9:1 according to training requirements. The training shard is used for local model training, and the validation shard is used for local model accuracy verification. Sensitive fields in the training shard are preprocessed using a combination of format anonymization and value anonymization. Among them, core equipment parameters and production formula data are replaced with the replacement method in value anonymization, replacing sensitive values ​​with encrypted virtual values. Confidential process parameters are masked using the masking method in format anonymization, hiding part of the content of sensitive fields to ensure that sensitive data is not leaked.

[0023] 4) Data Quality Verification Module: Establish a data quality verification index system, including data integrity, consistency, and accuracy. Integrity verification is achieved by determining whether data fields are missing, consistency verification is achieved by comparing data from different acquisition nodes of the same device, and accuracy verification is achieved by comparing data with a preset threshold range. The preprocessed industrial data is verified one by one, and data that fails verification is discarded to generate local training data that meets the requirements of federated training, ensuring the high quality of training data.

[0024] 2. Local Federal Training Layer The local federated training layer is deployed across various industrial participant nodes (such as different factories, different production workshops, and different equipment clusters). Each industrial participant node corresponds to a local federated training unit, which is used to complete personalized model training based on preprocessed local industrial data, avoiding privacy leaks caused by uploading raw data. It includes a model initialization module, a personalized training configuration module, a local training execution module, a training parameter extraction module, and a parameter preprocessing module. The specific functions of each module are as follows: 1) Model initialization module: Receives global initial model parameters issued by the federated collaborative scheduling layer. The global initial model parameters are preset based on industrial scenario requirements (such as equipment failure prediction and production quality prediction), adopt a deep learning model architecture, and are adapted to the time-series and high-dimensional characteristics of industrial big data. According to the type and scale of local industrial data, it completes the initialization configuration of the local model to ensure the consistency between the local model and the global initial model, laying the foundation for subsequent local training.

[0025] 2) Personalized Training Configuration Module: Based on the type (time-series data, static data), scale, and training objectives (accuracy requirements, inference speed requirements) of local industrial data, personalized training hyperparameters are configured. These hyperparameters include learning rate (adjustable from 0.001 to 0.1), number of iterations (adjustable from 100 to 1000 rounds), batch size (adjustable from 32 to 256), and regularization coefficient (adjustable from 0.0001 to 0.01). Dynamic adjustments are supported during the training process. Based on the accuracy feedback of each training round, the learning rate and regularization coefficient are automatically adjusted to improve the training effect of the local model.

[0026] 3) Local Training Execution Module: Employs an improved federated learning algorithm to perform iterative training of the local model based on local training data. This improved federated learning algorithm is based on the FedAvg algorithm and incorporates a time-series weight factor for industrial data, assigning different weights to different time segments of the time-series industrial data to improve the training adaptability of the time-series industrial data. A training interruption recovery unit is set up so that when the local training node experiences abnormal situations such as power outages or network interruptions that cause training interruptions, the current training state (including the number of iterations, current model parameters, and training loss value) and parameter information are saved. When the abnormality is recovered, training continues based on the saved training state without restarting the training process, thus improving training efficiency.

[0027] 4) Training Parameter Extraction Module: After each round of local training, the module extracts the parameter update amount of the local model. The parameter update amount includes the increment of weight parameters and the increment of bias parameters. No original local data is extracted to ensure the privacy and security of local data. The module records the parameter update amount and local model accuracy for each round of training for subsequent parameter uploading and training effect evaluation.

[0028] 5) Parameter preprocessing module: The extracted parameter update amounts are normalized and sparsified. The normalization process uses the Z-score normalization algorithm to map the parameter update amounts to a standard normal distribution, avoiding the impact of parameter scale differences on the aggregation effect. The sparsity process uses the L1 regularization algorithm to remove invalid parameter update amounts (parameters with absolute values ​​less than a preset threshold), reducing data transmission volume and privacy leakage risks, while improving subsequent aggregation efficiency.

[0029] 3. Federal Coordination and Scheduling Layer The federated collaborative scheduling layer is the core scheduling center of the system, used to realize the collaborative control of each local federated training node and aggregation node, optimize the allocation of training resources, and ensure the smooth progress of the global training process. It includes a collaborative control center, a node status monitoring module, a training task allocation module, a communication link management module, and a training progress synchronization module. The specific functions of each module are as follows: 1) Collaborative Control Center: As the core scheduling unit of the system, it adopts a distributed architecture to avoid single points of failure; it receives feedback information from each module (including node status, training progress, model accuracy, and privacy and security status), issues collaborative scheduling instructions, and coordinates the collaborative work of each module; it integrates a key management unit to generate, distribute, update, and destroy the encryption keys required by each module of the system. It adopts a secret sharing algorithm to achieve distributed key management, splitting and storing keys on multiple nodes to avoid the risk of privacy leakage caused by a single node holding all keys. The key update cycle can be dynamically adjusted according to the privacy protection level (adjustable from 1 to 7 days), without relying on a trusted third party to generate keys, reducing system deployment costs and security risks.

[0030] 2) Node Status Monitoring Module: Real-time monitoring of the computing resource utilization (including CPU utilization, GPU utilization, and memory utilization), network bandwidth, training progress, and node online status of each local federated training node, with a monitoring cycle of 10 seconds / time; generating a node status evaluation report, using a 100-point scale to score the performance of each node, with scoring indicators including computing resources (40 points), network status (30 points), and historical training efficiency (30 points), providing a basis for subsequent training task allocation.

[0031] 3) Training Task Allocation Module: Based on the node status evaluation report and training task requirements (such as training accuracy and training cycle), a dynamic task allocation algorithm is used to reasonably allocate global training tasks to each local federated training node to achieve load balancing. The dynamic task allocation algorithm is based on node performance score and task weight. The higher the node performance score, the greater the weight of the allocated training task. The task weight is positively correlated with the amount of training data and the number of iterations. When the performance of a node declines, the task allocation scheme is automatically adjusted to transfer some tasks to nodes with better performance to ensure training efficiency.

[0032] 4) Communication Link Management Module: Constructs encrypted communication links using the TLS 1.3 protocol, selects the optimal communication path (based on network latency and bandwidth stability); monitors the stability of the communication link in real time, and automatically switches to a backup communication link when abnormalities such as excessive latency (over 500ms) or interruption occur. The backup communication link uses different network operators and transmission paths to ensure the continuity and stability of data transmission; records the transmission log of the communication link, including transmission time, data volume, and transmission status, for subsequent troubleshooting.

[0033] 5) Training Progress Synchronization Module: This module synchronizes the training progress of each local federated training node with the collaborative control center, recording the number of iterations, parameter upload time, and training accuracy for each training round. When the training progress of a node lags behind the global average progress by more than 10 rounds, a progress adjustment instruction is issued to adjust the training hyperparameters of that node (increase the learning rate, increase the batch size) to accelerate the training progress and ensure consistent global training progress. Once all nodes have completed the current round of training, the next round of training instruction is triggered to ensure the orderly progress of the training process.

[0034] 4. Enhanced Privacy Protection Layer The privacy protection enhancement layer provides end-to-end privacy protection for data, parameters, and communication during training, resisting various attacks and ensuring the privacy and security of sensitive industrial data and model parameters. It includes a multi-layer encryption module, a differential privacy enhancement module, an access control module, and an attack detection module. The specific functions of each module are as follows: 1) Multi-layer encryption module: It adopts a three-level encryption architecture of "transmission encryption - storage encryption - computation encryption" to achieve full-process encryption protection; transmission encryption uses the TLS1.3 protocol to encrypt the parameter transmission process to ensure that the parameters are not stolen or tampered with during transmission; storage encryption uses the AES-256 algorithm to encrypt and store local training data and model parameters. The encryption key is distributed by the key management unit of the collaborative control center. Each node has an independent encryption key to ensure the privacy and security of the stored data; computation encryption uses the homomorphic encryption algorithm to encrypt the intermediate calculation results during the training process to achieve model training in ciphertext state and avoid privacy risks caused by the leakage of intermediate calculation results; the three-level encryption architecture is independent of each other and complements each other to form a comprehensive encryption protection system.

[0035] 2) Differential Privacy Enhancement Module: After extracting local model parameters, this module adds adaptive noise to the parameter update. The intensity of the adaptive noise is dynamically adjusted based on the local data scale and privacy protection level. The larger the local data scale and the lower the privacy protection level, the smaller the noise intensity, and vice versa. This ensures privacy and prevents excessive noise from affecting model accuracy. A dynamic privacy budget allocation algorithm is used to dynamically allocate the privacy budget based on the number of iterations in each training round, data sensitivity, and model accuracy requirements. The total privacy budget is fixed, and the more iterations and the higher the data sensitivity, the more privacy budget is allocated, achieving an optimal balance between privacy protection and model accuracy. The privacy budget allocation process is encrypted to ensure that the allocation strategy is not leaked.

[0036] 3) Access Control Module: A Role-Based Access Control (RBAC) mechanism is adopted, dividing access permissions into different levels, including system administrator, node administrator, training operator, and auditor. The access scope of each role to data, models, and parameters is clearly defined. The system administrator has the highest privileges and can manage all modules and nodes. Node administrators can only manage the relevant data and models of their corresponding local nodes. Training operators can only perform local training operations, and auditors can only view access logs and training logs. All access operations require authentication (using a triple authentication method of username + password + dynamic verification code), and all access operation logs are recorded, including access time, access role, access content, and operation result, for subsequent auditing and troubleshooting.

[0037] 4) Attack Detection Module: Monitors abnormal behaviors during the training process in real time, including parameter tampering attacks, malicious node attacks, and data theft attacks; uses an anomaly detection algorithm (optimized based on the Isolation Forest algorithm) to identify attack behaviors, and determines whether an attack exists by analyzing the integrity of parameter transmission, the standardization of node behavior, and the rationality of data access; when an attack is detected, it immediately blocks the attack (disconnects the communication link and isolates the attacking node), sends alarm information (including attack type, attack time, attacking node, and attack behavior description) to the collaborative control center, and records attack logs for subsequent attack tracing and protection optimization; performs security checks on isolated malicious nodes, and only allows them to reconnect to the system after confirming that there are no security risks.

[0038] 5. Model Aggregation and Validation Layer The model aggregation and validation layer is used to securely aggregate and verify the authenticity of model parameters uploaded from each local training node, generating a high-precision and reliable global model. It includes a parameter secure aggregation module, an aggregation result validation module, a model accuracy evaluation module, and a model optimization and adjustment module. The specific functions of each module are as follows: 1) Parameter Security Aggregation Module: Receives preprocessed parameter updates uploaded by each local federated training node and uses a weighted aggregation algorithm to perform global model parameter aggregation calculation. The weights of the weighted aggregation algorithm are determined comprehensively based on the training data scale, data quality, and training accuracy of each local node. The larger the training data scale, the higher the data quality, and the higher the training accuracy, the greater the weight, with a total weight of 1. The aggregation process uses encrypted calculation to ensure that the parameters are not leaked during the aggregation process. After aggregation is completed, global model parameters are generated for subsequent verification and distribution.

[0039] 2) Aggregation Result Verification Module: A bilinear aggregation signature algorithm is used to verify the authenticity and integrity of the aggregated global model parameters. Each local training node attaches its own digital signature when uploading parameter updates. After receiving the parameters, the aggregation node verifies the legality of the digital signature to ensure that the parameters come from legitimate local training nodes. After aggregation is completed, a verification signature of the aggregation result is generated based on the digital signatures of each local node and the aggregation algorithm. The verification signature is used to determine whether the aggregation result has been tampered with. If the verification is successful, the global model parameters are sent to each local training node. If the verification fails, the aggregation process is retried, and abnormal nodes are investigated.

[0040] 3) Model Accuracy Evaluation Module: This module uses preset accuracy evaluation metrics to test the accuracy of the aggregated global model. These metrics include accuracy, recall, F1 score, and time-series prediction error, with the time-series prediction error calculated using MAE (mean absolute error). A preset accuracy threshold (set according to industrial scenario requirements, such as accuracy ≥ 95%, MAE ≤ 0.05) is used. When the model accuracy reaches the preset threshold, global model training is confirmed to be complete. If the model accuracy does not reach the preset threshold, a retraining process is triggered, adjusting training hyperparameters and aggregation weights, and re-training and parameter aggregation are performed locally.

[0041] 4) Model Optimization and Adjustment Module: Based on the accuracy assessment results and feedback information during the training process (including the training accuracy of each node, parameter update volume, and noise impact), dynamically adjust the global training hyperparameters (learning rate, number of iterations, regularization coefficient) and aggregate weights; for nodes with low training accuracy, adjust their individual training hyperparameters and reduce their aggregate weights to avoid affecting the global model accuracy; optimize the training algorithm and dynamically adjust the time-series weight factor according to the characteristics of industrial data to improve the model's adaptability to industrial big data; continuously iterate and optimize until the global model accuracy reaches the preset threshold.

[0042] 6. Model Deployment Application Layer The model deployment and application layer is used to deploy the aggregated global model to industrial scenarios and realize application feedback, promoting the practical application of training results. It also provides data support for iterative model optimization. This includes a model conversion module, a deployment and adaptation module, a real-time inference module, and an application feedback module. The specific functions of each module are as follows: 1) Model Conversion Module: Converts the aggregated global model into a deployment format suitable for industrial scenarios, supporting multi-terminal deployment of edge computing devices (such as industrial gateways and edge servers), industrial servers, and cloud platforms; deployment formats include ONNX, TensorRT, PB, etc. The module automatically selects the optimal deployment format based on the hardware configuration and software environment of the deployment device to ensure model deployment compatibility; during the conversion process, the model is lightweighted, redundant parameters are removed, and the model size and resource consumption are reduced.

[0043] 2) Deployment and Adaptation Module: Based on the hardware configuration (CPU, GPU, memory) and application requirements (inference speed, accuracy requirements) of different industrial scenarios, the deployed model is personalized and adapted; for edge computing devices, the model inference speed is optimized, resource consumption is reduced, and inference latency is ensured to be no more than 50ms; for industrial servers and cloud platforms, the model accuracy is optimized and inference stability is improved; seamless integration between the model and industrial equipment, SCADA system, and MES system is achieved, supporting real-time data interaction and command issuance.

[0044] 3) Real-time inference module: Receives real-time data from industrial scenarios (collected and preprocessed in real-time by the industrial data access layer), performs real-time inference through the deployed global model, and outputs inference results. The inference results include equipment failure prediction (predicting failure type, failure time, and failure location), production quality prediction (predicting product pass probability and quality defect type), and process parameter optimization suggestions (specific values ​​of process parameters to be adjusted and the timing of adjustment). The inference results are displayed in a visual manner (charts and text prompts) and simultaneously distributed to relevant industrial equipment and systems to guide production operations.

[0045] 4) Application Feedback Module: Collects inference results, accuracy performance (deviation between actual accuracy and preset threshold) and changes in industrial scenario requirements (such as new fault types or process adjustments) during model application, and generates an application feedback report; feeds the application feedback report back to the federated collaborative scheduling layer and the model aggregation and verification layer to provide data support for iterative optimization of the model; when the application scenario changes significantly or the model accuracy drops by more than 5%, the model retraining process is triggered to ensure that the model always adapts to the needs of the industrial scenario.

[0046] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0047] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A federated learning model collaborative training and privacy protection system for industrial big data, characterized in that, It includes an industrial data access layer, a local federated training layer, a federated collaborative scheduling layer, a privacy protection enhancement layer, a model aggregation and verification layer, and a model deployment and application layer. Each layer works together from top to bottom to achieve privacy security and efficient collaborative training of industrial big data. The industrial data access layer is used to complete the distributed access and standardized preprocessing of multi-source heterogeneous industrial data. The local federated training layer is used to complete personalized model training based on preprocessed local industrial data. The federated collaborative scheduling layer is used to realize the collaborative control of each local training node and the aggregation node; The privacy protection enhancement layer is used to provide full-process privacy protection for data, parameters and communication processes during training. The model aggregation and verification layer is used to perform secure aggregation and authenticity verification of the model parameters uploaded by each local training node; The model deployment application layer is used to deploy the aggregated global model to industrial scenarios and realize application feedback.

2. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 1, characterized in that: The industrial data access layer includes a multi-source heterogeneous data access module, a data cleaning and standardization module, a data fragmentation and desensitization module, and a data quality verification module. The multi-source heterogeneous data access module adopts a multi-protocol adaptation architecture, supports distributed access of industrial sensors, PLCs, SCADA systems, MES systems and industrial databases, and is compatible with multiple industrial communication protocols such as Modbus, OPC UA, MQTT and HTTP, enabling real-time acquisition of time-series monitoring data, equipment operation data, production process data and environmental parameter data. The data cleaning and standardization module is used to remove outliers, fill in missing values, unify data formats, and normalize dimensions of the collected industrial big data to generate standardized industrial datasets. The data segmentation and desensitization module is used to divide the standardized industrial dataset into training segments and validation segments according to preset segmentation rules. Sensitive fields in the training segments are preprocessed using a combination of format desensitization and value desensitization. The sensitive fields include core equipment parameters, production formula data, and confidential process parameters. The data quality verification module is used to verify the integrity, consistency, and accuracy of the preprocessed industrial data, discard unqualified data, and generate local training data that meets the requirements of federated training.

3. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 1, characterized in that: The local federated training layer is deployed on each industrial participant node, and each industrial participant node corresponds to a local federated training unit. The local federated training unit includes a model initialization module, a personalized training configuration module, a local training execution module, a training parameter extraction module, and a parameter preprocessing module. The model initialization module is used to receive global initial model parameters issued by the federated collaborative scheduling layer and complete the initialization configuration of the local model. The global initial model parameters are preset based on the needs of industrial scenarios and are adapted to the time-series and high-dimensional characteristics of industrial big data. The personalized training configuration module is used to configure personalized training hyperparameters according to the type, scale and training objectives of local industrial data. The hyperparameters include learning rate, number of iterations, batch size and regularization coefficient, and support dynamic adjustment during the training process. The local training execution module adopts an improved federated learning algorithm to complete the iterative training of the local model based on local training data. The improved federated learning algorithm is based on the FedAvg algorithm optimization and introduces the time series weight factor of industrial data to improve the training adaptability of time series industrial data. The training parameter extraction module is used to extract the parameter update amount of the local model after each round of local training. The parameter update amount includes the weight parameter increment and the bias parameter increment. The parameter preprocessing module is used to normalize and sparsify the extracted parameter update amounts, remove invalid parameter update amounts, and reduce data transmission volume and privacy leakage risks.

4. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 1, characterized in that: The federal collaborative scheduling layer includes a collaborative control center, a node status monitoring module, a training task allocation module, a communication link management module, and a training progress synchronization module. The collaborative control center, as the core scheduling unit of the system, is used to receive feedback information from modules at each layer, issue collaborative scheduling instructions, and coordinate the collaborative work of each module. The node status monitoring module is used to monitor the computing resource utilization, network bandwidth, training progress and node online status of each local federated training node in real time, and generate a node status evaluation report. The computing resource utilization includes CPU utilization, GPU utilization and memory utilization. The training task allocation module is used to allocate global training tasks to each local federated training node in a reasonable manner according to the node status evaluation report and training task requirements, using a dynamic task allocation algorithm to achieve load balancing. The dynamic task allocation algorithm completes the allocation based on node performance score and task weight. The node performance score is determined by a combination of computing resources, network status and historical training efficiency. The communication link management module is used to build encrypted communication links, select the optimal communication path, monitor the stability of the communication links, and automatically switch to the backup communication link when the link is abnormal to ensure the continuity of data transmission. The training progress synchronization module is used to synchronize the training progress of each local federated training node with the collaborative control center. It records the number of iterations, parameter upload time and training accuracy for each round of training. When the training progress of a certain node lags behind, it issues a progress adjustment instruction to ensure that the global training progress is consistent.

5. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 1, characterized in that: The privacy protection enhancement layer includes a multi-layer encryption module, a differential privacy enhancement module, an access control module, and an attack detection module; The multi-layer encryption module adopts a three-level encryption architecture of "transmission encryption - storage encryption - computation encryption". Transmission encryption uses the TLS1.3 protocol to encrypt the parameter transmission process, storage encryption uses the AES-256 algorithm to encrypt and store local training data and model parameters, and computation encryption uses the homomorphic encryption algorithm to encrypt the intermediate calculation results during the training process, so as to realize model training in ciphertext state. The differential privacy enhancement module is used to add adaptive noise to the parameter update after the local model parameters are extracted. The intensity of the adaptive noise is dynamically adjusted according to the local data scale and privacy protection level, so as to ensure privacy and security while avoiding excessive noise from affecting the model accuracy. The access control module adopts a role-based access control mechanism, which divides access permissions into different levels, clarifies the access scope of each role to data, models and parameters, and only users with corresponding permissions can perform relevant operations. At the same time, all access operation logs are recorded for subsequent auditing. The attack detection module is used to monitor abnormal behaviors during the training process in real time, including parameter tampering attacks, malicious node attacks, and data theft attacks. It uses anomaly detection algorithms to identify attack behaviors. When an attack is detected, it immediately blocks the attack, isolates malicious nodes, and sends alarm information to the collaborative control center.

6. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 1, characterized in that: The model aggregation and verification layer includes a parameter security aggregation module, an aggregation result verification module, a model accuracy evaluation module, and a model optimization and adjustment module. The parameter security aggregation module is used to receive the preprocessed parameter update amount uploaded by each local federated training node, and to complete the aggregation calculation of global model parameters using a weighted aggregation algorithm. The weights of the weighted aggregation algorithm are determined comprehensively based on the training data scale, data quality and training accuracy of each local node, so as to realize the optimization and update of the global model. The aggregation result verification module is used to verify the authenticity and integrity of the aggregated global model parameters using a bilinear aggregation signature algorithm, ensuring that the aggregation result has not been tampered with and that the parameters come from legitimate local training nodes. After the verification is passed, the global model parameters are sent to each local training node. If the verification fails, the aggregation process is retried. The model accuracy evaluation module is used to perform accuracy testing on the aggregated global model using preset accuracy evaluation indicators. The accuracy evaluation indicators include accuracy, recall, F1 score and time series prediction error. When the model accuracy does not reach the preset threshold, the retraining process is triggered. The model optimization and adjustment module is used to dynamically adjust the global training hyperparameters and aggregate weights based on the accuracy evaluation results and feedback information during the training process, thereby optimizing the training algorithm and improving the adaptability and accuracy of the global model.

7. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 1, characterized in that: The model deployment application layer includes a model conversion module, a deployment adaptation module, a real-time inference module, and an application feedback module; The model conversion module is used to convert the aggregated global model into a deployment format adapted to industrial scenarios, supporting multi-terminal deployment on edge computing devices, industrial servers and cloud platforms; The deployment and adaptation module is used to personalize the deployed model according to the hardware configuration and application requirements of different industrial scenarios, adjust the model inference speed and resource utilization, and achieve seamless connection between the model and industrial equipment and systems. The real-time inference module is used to receive real-time data from industrial scenarios, perform real-time inference through the deployed global model, and output inference results, including equipment failure prediction, production quality prediction, and process parameter optimization suggestions. The application feedback module is used to collect inference results, accuracy performance and changes in industrial scenario requirements during the model application process, generate application feedback reports, and feed them back to the federated collaborative scheduling layer and the model aggregation and verification layer to provide data support for the iterative optimization of the model.

8. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 3, characterized in that: The local training execution module also includes a training interruption recovery unit, which is used to save the current training state and parameter information when the training is interrupted due to abnormal situations such as power failure or network interruption of the local training node. When the abnormality is recovered, the training can continue to be completed based on the saved training state without restarting the training process, thereby improving training efficiency.

9. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 4, characterized in that: The collaborative control center also includes a key management unit, which is used to generate, distribute, update and destroy the encryption keys required by each module of the system. It adopts a secret sharing algorithm to realize distributed key management, avoid the risk of privacy leakage caused by a single node holding all keys, and the key update cycle can be dynamically adjusted according to the privacy protection level.

10. The federated learning model collaborative training and privacy protection system for industrial big data according to claim 5, characterized in that: The differential privacy enhancement module employs a dynamic privacy budget allocation algorithm, which dynamically allocates the privacy budget based on the number of iterations in each training round, data sensitivity, and model accuracy requirements, achieving an optimal balance between privacy protection and model accuracy. The privacy budget allocation process is encrypted to ensure that the allocation strategy is not leaked.

Images