An industrial multi-source heterogeneous data instant general fusion model training method and device

Abstract

This application provides a model training method and apparatus for real-time generalized fusion of multi-source heterogeneous data in industrial applications, relating to the field of model training technology. It is used to efficiently and stably collaboratively train models located on different nodes for heterogeneous data across different nodes, resulting in a model with superior performance. The method includes: determining the target loss based on local training samples on the target node and the target model deployed on the target node; obtaining the model parameters of the models to be trained in the neighboring nodes of the target node; determining the target difference value based on the model parameters of the models to be trained and the model parameters of the target model; determining the momentum of the current iteration based on the target loss, the target difference value, and the momentum of the target model in the previous iteration; the momentum is used to characterize the update direction and magnitude of the model parameters; and updating the target model based on the momentum of the current iteration.

Description

Technical Field

[0001] This application relates to the field of model training technology, and in particular to a model training method and apparatus for real-time generalized fusion of multi-source heterogeneous industrial data. Background Technology

[0002] In multi-node network scenarios, a large amount of data is generated and stored locally by nodes such as terminal devices, edge servers, and other organizations. In order for the models in each node to perform the same task, it is necessary to use the data generated by each node for model training. If only the data generated locally by the node is used for model training, the model performance will be poor and unable to cope with different situations of the same task.

[0003] To collaboratively train models using data from multiple nodes, a decentralized, peer-to-peer collaborative training scheme is currently employed, where nodes only communicate with their neighbors to exchange model information. However, due to differences in construction time, technical standards, business needs, and the varying brands and models of production equipment, as well as different data collection and storage methods, it is difficult to unify the data across different nodes. Current solutions struggle to effectively coordinate the differences in model update directions caused by heterogeneous data distribution, easily leading to training process oscillations, slow convergence, and even model divergence across nodes, failing to achieve effective global consensus. Summary of the Invention

[0004] This application provides a model training method and apparatus for real-time generalized fusion of industrial multi-source heterogeneous data, which is used to efficiently and stably train models located in different nodes for heterogeneous data between different nodes, so as to obtain models with better performance.

[0005] To achieve the above objectives, this application adopts the following technical solution: In the first aspect, this application provides a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, which is applied to a target node, where the target node is any node in a target network containing multiple nodes; and each of the multiple nodes has a model to be trained deployed to perform the same task.

[0006] The method includes: The target loss is determined based on the local training samples of the target node and the target model deployed in the target node; Obtain the model parameters of the model to be trained from the neighboring nodes of the target node; Based on the model parameters of the model to be trained and the model parameters of the target model, determine the target difference value; Based on the target loss, target difference value, and the momentum of the target model in the previous iteration, the momentum of the current iteration is determined; the momentum is used to characterize the direction and magnitude of the model parameter update. The target model is updated based on the momentum generated in this iteration.

[0007] The technical solution provided in this application brings at least the following beneficial effects: During model training, each node acquires and references model parameters from neighboring nodes without sharing local training samples. This allows for collaborative improvement of model capabilities while protecting local data privacy. Furthermore, since only model parameters are transmitted between nodes, the amount of data exchanged and the bandwidth requirements are reduced.

[0008] The target loss represents the local data-driven optimization direction of a node, the target difference value represents the globally consistent and coordinated optimization direction from neighboring nodes, and the historical momentum represents the inertial optimization direction of the training process. The target loss, target difference value, and historical momentum together determine the momentum that characterizes the update direction and magnitude of model parameters. In other words, the requirement for consistent collaborative optimization participates in and influences the model optimization trajectory of each node from the outset. This effectively alleviates the problem of model divergence and training oscillation caused by heterogeneous data between different nodes, reconciles the contradiction between a node's own optimization objective and the model state of neighboring nodes, and guides nodes to spontaneously converge towards the network consensus while pursuing their own optimal performance, thereby significantly improving training stability and overall convergence efficiency. Consistent collaborative optimization allows the model to learn the detection capabilities of other nodes in different situations without transmitting raw data between nodes, ultimately resulting in a better-performing model.

[0009] Optionally, the momentum of the current iteration is determined based on the target loss, the target difference value, and the momentum of the target model in the previous iteration, including: determining the target gradient based on the target loss; the target gradient is used to characterize the adjustment direction of the model parameters that reduce the target loss; determining the dual variable based on the target difference value; the dual variable is used to characterize the degree of difference in model parameters between the target node and its neighboring nodes; and determining the momentum of the current iteration based on the target gradient, the dual variable, and the momentum of the target model in the previous iteration.

[0010] Optionally, based on the target difference value, the dual variable is determined, including: determining the penalty parameter; the penalty parameter is used to characterize the contribution of the target difference value to the dual variable; and determining the dual variable for the current iteration based on the dual variable, the target difference value, and the penalty parameter from the previous iteration.

[0011] Optionally, the dual variable in this iteration process satisfies the following relationship with the dual variable, target difference value, and penalty parameter in the previous iteration process: ;in, Used to represent the dual variables of node m and node n in the (k+1)th iteration. This is used to represent the dual variable between node m and node n in the k-th iteration, where k is a natural number. Used to represent penalty parameters Used to represent the model parameters of node n in the (k+1)th iteration. Used to represent the model parameters of node m during the (k+1)th iteration. Used to represent the target difference value.

[0012] Optionally, the momentum in this iteration process satisfies the following relationship with the target gradient, the dual variable, and the momentum of the target model in the previous iteration process: ;in, Used to represent the momentum of node n in the (e+1)th iteration. Used to represent the target gradient. Used to represent dual variables, Used to represent the momentum of the target model in the previous iteration. Used to represent penalty parameters Used to represent the target difference value. Used to represent the momentum parameter, where e is a natural number.

[0013] Optionally, the model parameters of the model to be trained in the neighboring nodes of the target node are obtained, including: obtaining the model parameters of the model to be trained in the neighboring nodes of the target node according to a preset period, wherein the preset period includes a preset number of iterations.

[0014] Secondly, this application provides a model training device for real-time generalized fusion of multi-source heterogeneous industrial data, comprising: The processing module is used to determine the target loss based on the training samples local to the target node and the target model deployed in the target node; The acquisition module is used to acquire the model parameters of the model to be trained in the neighboring nodes of the target node; The processing module is also used to determine the target difference value based on the model parameters of the model to be trained and the model parameters of the target model. The processing module is also used to determine the momentum of the current iteration based on the target loss, target difference value, and the momentum of the target model in the previous iteration; the momentum is used to characterize the update direction and magnitude of the model parameters. The processing module is also used to update the target model based on the momentum of the current iteration.

[0015] Thirdly, this application provides an electronic device, including: a processor and a memory; the memory is used to store computer execution instructions, the processor is connected to the memory, and when the electronic device is running, the processor executes the computer execution instructions stored in the memory to enable the electronic device to execute any of the above-mentioned industrial multi-source heterogeneous data real-time generalized fusion model training methods.

[0016] Fourthly, this application provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform any of the above-mentioned methods for real-time generalized fusion of multi-source heterogeneous industrial data.

[0017] Fifthly, this application provides a computer program product containing computer execution instructions, which, when run on a computer, causes the computer to execute any of the above-mentioned methods for real-time generalized fusion of multi-source heterogeneous industrial data for model training.

[0018] In the specific implementation of this application, the names of the components of the aforementioned device do not limit the device itself. In actual implementation, these components may appear under other names. As long as the function of each component is similar to the specific implementation of this application, it falls within the scope of the claims of this application and its equivalents.

[0019] Furthermore, the technical effects of any of the design methods in aspects two through five can be found in the technical effects of the different design methods in aspect one above, and will not be repeated here. Attached Figure Description

[0020] Figure 1 This application provides an illustration of a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, as provided in this embodiment. Figure 1 ; Figure 2 A schematic diagram of the target node structure used in a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, provided in an embodiment of this application; Figure 3 A flowchart illustrating a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, provided in an embodiment of this application; Figure 4 This application provides an illustration of a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, as provided in this embodiment. Figure 2 ; Figure 5 This application provides an illustration of a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, as provided in this embodiment. Figure 3 ; Figure 6This application provides an illustration of a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, as provided in this embodiment. Figure 4 ; Figure 7 A schematic diagram of the overall logic of a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, provided in an embodiment of this application; Figure 8 A schematic diagram of the structure of a model training device for real-time generalized fusion of multi-source heterogeneous data in industry, provided in an embodiment of this application; Figure 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the hardware structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0023] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0024] In modern industrial networks, for ease of management, vertical layering is typically used to form a hierarchical node network based on functional coupling and data interaction frequency. For example... Figure 1 As shown, in a network scenario composed of multiple nodes, the network includes 3 nodes at the enterprise layer (upstream enterprise, midstream enterprise and downstream enterprise), 3 nodes at the factory layer (factory A, factory B and factory C) and 3 nodes at the workshop layer (workshop 1, workshop 2 and workshop 3).

[0025] As the supply chain collaboration hub, the enterprise layer focuses on cross-organizational resource optimization (such as order allocation and capacity matching), achieving global scheduling through shared data on orders, inventory, and logistics. Data sharing among enterprises in the enterprise layer is crucial for achieving efficient model collaboration; however, data sharing is prone to privacy and security risks such as data leakage and misuse. This contradiction between privacy and security and collaboration efficiency creates access barriers for data sharing among enterprises.

[0026] The plant layer is responsible for production execution and equipment integration. Nodes typically correspond to physically isolated manufacturing bases, requiring solutions for protocol compatibility and data standardization across multiple generations of equipment (such as traditional machine tools and intelligent robots). Furthermore, due to differences in construction time, technical standards, and business needs, the plants within the plant layer have different brands and models of production equipment, varying data collection and storage methods, high communication protocol conversion costs, and a coexistence of structured, semi-structured, and unstructured data, making unified processing difficult. Moreover, industrial production processes are characterized by rapidly changing production states and frequent unexpected events such as equipment failures and process deviations. This necessitates a data fusion system capable of acquiring, processing, and analyzing various types of workshop data in real time, providing timely decision-making support for production scheduling and quality control. However, existing workshop data transmission links and processing platforms often suffer from data transmission delays and slow processing speeds when dealing with massive amounts of multi-source heterogeneous data, failing to meet the stringent real-time response requirements of industrial production.

[0027] It is evident that deep-seated contradictions exist in data flow among multiple nodes in the industrial sector: open sharing is a prerequisite for improving collaborative efficiency, but security risks limit the scope of data sharing; standardized integration requires a unified technological foundation, but path dependencies of legacy systems make integration difficult; real-time response relies on powerful computing capabilities, but massive amounts of data often exceed the limits of existing processing capacity. Current decentralized model collaborative training schemes struggle to effectively coordinate differences in model update directions caused by heterogeneous data distribution, easily leading to training process oscillations, decreased convergence speed, and even model divergence across nodes, failing to form effective global consensus.

[0028] To address the aforementioned issues, this application provides a model training method for real-time generalized fusion of multi-source heterogeneous industrial data, comprising: determining a target loss based on local training samples of the target node and a target model deployed in the target node; obtaining model parameters of the models to be trained in neighboring nodes of the target node; determining a target difference value based on the model parameters of the models to be trained and the model parameters of the target model; determining the momentum of the current iteration based on the target loss, the target difference value, and the momentum of the target model in the previous iteration; the momentum is used to characterize the update direction and magnitude of the model parameters; and updating the target model based on the momentum of the current iteration.

[0029] During model training, each node acquires and references model parameters from neighboring nodes without sharing local training samples. This allows for collaborative improvement of model capabilities while protecting local data privacy. Furthermore, since only model parameters are transmitted between nodes, the amount of data exchanged and the bandwidth requirements are reduced.

[0030] The target loss represents the local data-driven optimization direction of a node, the target difference value represents the globally consistent and coordinated optimization direction from neighboring nodes, and the historical momentum represents the inertial optimization direction of the training process. The target loss, target difference value, and historical momentum together determine the momentum that characterizes the direction and magnitude of model parameter updates. In other words, the requirement for consistent and coordinated optimization participates in and influences the model optimization trajectory of each node from the very beginning. This effectively alleviates the problem of model divergence and training oscillations caused by heterogeneous data between different nodes, coordinates the contradiction between a node's own optimization objective and the model state of neighboring nodes, and guides nodes to spontaneously converge towards the network consensus while pursuing their own optimal performance, thereby significantly improving training stability and overall convergence efficiency.

[0031] The model training method for real-time general fusion of multi-source heterogeneous industrial data provided in this application can be applied to target nodes, which are any nodes in a target network containing multiple nodes. Each of the multiple nodes deploys a model to be trained to perform the same task; the model to be trained in the target node is called the target model.

[0032] The target network can be an Industrial Internet of Things (IIoT), an edge computing network, a cross-institutional data collaboration network, a self-organizing network composed of terminal device clusters, or a distributed computing cluster in cloud computing. For example, when the target network is an IIoT, its nodes can be intelligent devices in a factory (such as machine tools, robots, etc.), industrial control computers on a production line, edge servers in the factory area, or data centers in different factory locations. As another example, when the target network is a cross-institutional data collaboration network, its nodes can be financial institutions (such as different banks), medical institutions (such as different hospitals), or logistics companies. Yet another example is a self-organizing network composed of terminal device clusters, where nodes can be smartphones, autonomous vehicles, drones, etc. This application does not impose any limitations on these examples.

[0033] The target node can be a computing device that can deploy a model and perform detection tasks through the model, such as a terminal device or a server.

[0034] The target node is used to: determine the target loss based on the local training samples and the target model deployed in the target node; obtain the model parameters of the models to be trained in the neighboring nodes of the target node; determine the target difference value based on the model parameters of the models to be trained and the model parameters of the target model; determine the momentum of the current iteration based on the target loss, the target difference value, and the momentum of the target model in the previous iteration; the momentum is used to characterize the update direction and magnitude of the model parameters; and update the target model based on the momentum of the current iteration.

[0035] Specifically, based on the target loss, target difference value, and the momentum of the target model in the previous iteration, the momentum of the current iteration is determined, including: determining the target gradient based on the target loss; the target gradient is used to characterize the direction of adjustment of model parameters that reduces the target loss; determining the dual variable based on the target difference value; the dual variable is used to characterize the degree of difference in model parameters between the target node and its neighboring nodes; and determining the momentum of the current iteration based on the target gradient, dual variable, and the momentum of the target model in the previous iteration.

[0036] Specifically, based on the target difference value, the dual variable is determined, including: determining the penalty parameter; the penalty parameter is used to characterize the contribution of the target difference value to the dual variable; based on the dual variable, target difference value and penalty parameter in the previous iteration process, the dual variable for the current iteration process is determined.

[0037] Based on the above description, such as Figure 2 As shown, the target node can be divided into a system layer and an algorithm layer. The system layer includes a local training module and a global coordination module. The local training module can be further divided into a local data storage submodule and a local model training submodule. The global coordination module can be further divided into a secure aggregation coordination submodule and a global parameter caching submodule. The algorithm layer includes a local iteration module and a global fusion module. The local iteration module can be further divided into a historical momentum caching submodule, a momentum calculation submodule, a local parameter update submodule, and a local model evaluation submodule. The global fusion module can be further divided into a dual variable management submodule, a penalty term calculation submodule, a global parameter consistency verification submodule, and a global parameter optimization submodule.

[0038] The local data storage submodule stores the raw local data, providing local data for the local model training submodule. For example, if the target node is a factory, the node's local data could be the operating data of the equipment in the factory (such as sensor data on vibration, temperature, and operating time).

[0039] The local model training submodule calls the local iteration module of the algorithm layer to start the model training process based on local data.

[0040] The global coordination module is responsible for cross-domain coordination: the secure aggregation coordination submodule collects model parameters from each adjacent node and performs secure aggregation using techniques such as encryption and differential privacy to prevent the parameters of a single node from being reverse-derived. The secure aggregation coordination submodule then passes the aggregated global parameters to the global parameter caching submodule, which temporarily stores the aggregated global parameters and passes them to the global fusion module in the algorithm layer. This allows the global fusion module to optimize the model parameters of the local target model based on the aggregated global parameters, and then sends the optimized model parameters to the local model training submodule.

[0041] The historical momentum cache submodule records the historical momentum generated during local model training, providing historical momentum for the momentum calculation submodule.

[0042] The momentum calculation submodule combines historical momentum and target gradient to calculate and output the momentum calculated in the current iteration.

[0043] The local parameter update submodule uses momentum to adjust the model parameters of the target model and passes the updated model parameters to the local model evaluation submodule. This achieves fast and stable local convergence, helping the target model learn detection patterns more quickly on local data. For example, it helps the model learn the fault patterns corresponding to abnormal equipment vibration more quickly based on the operating data of equipment in a factory.

[0044] The local model evaluation submodule evaluates the model performance on the local validation set and determines whether the model meets the local convergence requirements.

[0045] The dual variable management submodule calculates the differences in model parameters between the target node and its neighboring nodes, and then calculates the dual variable based on these differences. This is the standard for measuring parameter drift caused by heterogeneous data.

[0046] The penalty term calculation submodule calculates and outputs the penalty term based on the size of the dual variable. The greater the difference, the stronger the penalty term, forcing the model parameters of each node to adjust in a globally consistent direction.

[0047] The global parameter consistency verification submodule checks whether the model parameters adjusted by the alternating direction multiplier method (ADMM) meet the consistency threshold and feeds back the consistency results to the global parameter optimization submodule.

[0048] The global parameter optimization submodule optimizes model parameters using the ADMM algorithm based on dual variables and penalty terms, ensuring that the models of the target node and its neighboring nodes maintain global semantic consistency while preserving the personalized features of the node's local data. The optimized model parameters are then sent back to the global parameter caching submodule.

[0049] Specifically, the interactions between the various modules of the target node are as follows: Local training phase: The local model training submodule of the target node calls the local iteration module. Based on the original data stored in the local data storage submodule, it completes the initial training of the target model through the process of "historical momentum cache → momentum calculation → local parameter update → local model evaluation".

[0050] Cross-domain collaboration phase: After obtaining the model parameters of adjacent nodes, they are uploaded to the secure aggregation and coordination submodule. After aggregation, they are passed to the global parameter cache submodule and then enter the global fusion module. Through the process of "dual variable calculation → penalty term calculation → global parameter optimization → consistency verification", the globally optimized model parameters are obtained.

[0051] Iterative Loop Phase: The global parameter caching submodule sends the optimized parameters back to the local model training submodule of the target node, starting the next round of "local training → cross-domain collaboration" iteration until the model meets the requirements in terms of local performance and global consistency.

[0052] This modular division of labor and interaction not only ensures the privacy and security of data by keeping it within the local machine, but also solves the problem of difficult heterogeneous data fusion, improves training convergence speed, and achieves efficient collaborative modeling.

[0053] The following is a detailed description of the model training method for real-time generalized fusion of multi-source heterogeneous industrial data provided in the embodiments of this application, with reference to the accompanying drawings.

[0054] Figure 3 This is a flowchart illustrating a model training method for real-time generalized fusion of multi-source heterogeneous data in industry, provided in an embodiment of this application. The method can be executed by the aforementioned target node and includes: S101. Determine the target loss based on the local training samples of the target node and the target model deployed in the target node.

[0055] Training samples should include sample features and their corresponding ground truth labels. For example, a sample feature might be a device sensor reading, and its corresponding ground truth label might be the device's normal or faulty state.

[0056] In actual training, there are multiple training samples.

[0057] The target model can be any machine learning model whose parameters can be optimized using gradient descent. Examples include convolutional neural networks for image recognition, recurrent neural networks for time series prediction, fully connected neural networks for structured data classification, and autoencoders for anomaly detection.

[0058] The specific tasks performed by the target model can be: anomaly detection (identifying whether the device's operating status deviates from the normal mode), fault diagnosis (diagnosing the location of the device malfunction), network intrusion detection (identifying network attacks, malware, or abnormal access behavior), and detecting the target's entry into the area.

[0059] Local training samples are input into the target model, which calculates predictions based on the current model parameters and outputs the predicted results. The predictions of the target model are compared with the true labels, and the loss for each training sample is calculated based on a preset loss function. Finally, the average loss of the current batch of local training samples is determined as the target loss. The target loss quantifies the overall prediction error of the current model parameters on local data.

[0060] The preset loss function can be a loss function pre-selected by the user that matches the task being performed. For example, the cross-entropy loss function is often used for classification tasks, while the mean squared error loss function is often used for regression tasks.

[0061] The architecture of the models to be trained deployed in each node of the target network can be the same, or they can be heterogeneous due to differences in local computing resources or data characteristics, as long as the models can achieve coordination at the parameter space or functional level.

[0062] S102. Obtain the model parameters of the model to be trained in the neighboring nodes of the target node.

[0063] Specifically, the target node obtains the model parameters of the model to be trained from the neighboring nodes of the target node according to a preset period, which includes a preset number of iterations.

[0064] For example, the preset period can be 5, that is, the model parameters of adjacent nodes are obtained once every 5 iterations.

[0065] In the following description, an iteration process of one cycle is called a global iteration process, and each iteration process within each cycle is called a local iteration process. That is, the model undergoes K rounds of global iteration, and each round of global iteration includes E local iteration processes. K and E are both positive integers.

[0066] Before the first iteration begins, the model parameters of the models to be trained at each node are initialized. The initialized model parameters can be random, or they can be the model parameters of a pre-loaded task-related pre-trained model, or they can be a preset initial model parameter.

[0067] At the beginning of each global iteration, a node exchanges the latest model parameters with its neighboring nodes. During each local iteration of this global iteration, the model parameters of the neighboring nodes used are the model parameters of the neighboring nodes obtained at the beginning of this global iteration.

[0068] The number of local iterations can be dynamically adjusted according to the degree of data heterogeneity, and momentum states can be passed across rounds to accelerate convergence, achieving a balance between communication overhead and communication efficiency.

[0069] Adjacent nodes are nodes that have a direct communication channel with the target node. For example, with... Figure 1 For example, if the upstream enterprise directly communicates with plant A, plant B, and plant C, communicates with the midstream enterprise, and does not directly communicate with the downstream enterprise, then the downstream enterprise is not an adjacent node of the upstream enterprise. If plant A, plant B, and plant C do not communicate with the midstream or downstream enterprises, then the midstream and downstream enterprises are not adjacent nodes of any plant node.

[0070] The target node obtains the model parameters of all its neighboring nodes. For example, with Figure 1 For example, if factory area B communicates with workshops 1, 2 and 3, and factory area B communicates with factory area A and factory area C, then the adjacent nodes of factory area B include upstream enterprise, factory area A, factory area C, workshop 1, workshop 2 and workshop 3. When factory area B wants to obtain the model parameters of its adjacent nodes, it needs to obtain the model parameters of the upstream enterprise, factory area A, factory area C, workshop 1, workshop 2 and workshop 3.

[0071] If every node needs to obtain parameters from other nodes in the target network, a massive number of communication requests will be generated. When the number of nodes in the target network is large, this will cause network congestion, leading to a sharp increase in synchronization latency. Furthermore, if a node can obtain the model parameters of all nodes, once that node is compromised or maliciously penetrated, the attacker can obtain information from the entire network, resulting in a large-scale privacy breach.

[0072] This application only obtains model parameters from adjacent nodes, resulting in low communication load and low real-time latency. Sensitive parameter information is only disseminated among a limited number of trusted neighbors. Even if a node is compromised, the impact is limited to its immediate neighbors, preventing access to the entire network and significantly improving overall privacy and security. When a new node is added to the target network, it only needs to establish connections with a few physical or logical neighbors to join the global collaborative model optimization, with minimal impact on the entire network. Even with communication only with adjacent nodes, consensus information can propagate through multiple iterations, like ripples, ultimately enabling the entire network to reach consensus.

[0073] S103. Based on the model parameters of the model to be trained and the model parameters of the target model, determine the target difference value.

[0074] The target difference value characterizes the degree of inconsistency between the model states of the target node and its neighboring nodes. The target difference value can be a difference vector or scalar calculated by directly subtracting the model parameters of the target model from the model parameters of the training models of its neighboring nodes, or by comparison. For example, in the e-th round of local iteration, the model parameters of the target node are... The model parameters of adjacent nodes are The target difference value is .

[0075] S104. Based on the target loss, target difference value, and the momentum of the target model in the previous iteration, determine the momentum of the current iteration.

[0076] Momentum is used to characterize the direction and magnitude of the model parameter update.

[0077] Specifically, the target node determines the target gradient based on the target loss; the target gradient is used to characterize the direction of adjustment of model parameters that reduces the target loss; then, based on the target difference value, the dual variable is determined; the dual variable is used to characterize the degree of difference in model parameters between the target node and its neighboring nodes; finally, based on the target gradient, the dual variable, and the momentum of the target model in the previous iteration, the momentum of the current iteration is determined.

[0078] Before the first iteration begins, the dual variables corresponding to each neighboring node are also initialized. The initial values ​​of the dual variables are... =0 (the dual variable between the target node n and the neighboring node m is 0 before the first iteration). The dual variable represents the Lagrange multiplier introduced in the ADMM algorithm, which is used to handle the consistency constraint in distributed optimization. The consistency constraint in this application is that the model parameters of the neighboring node and the target node must be consistent.

[0079] Since the model parameters are acquired periodically, the target difference value and the dual variable calculated based on the target difference value are the same in each local iteration of a global iteration.

[0080] The target gradient and the target loss satisfy the following relationship: ; in, This represents the target gradient of target node n in the e-th local iteration. This represents the gradient of the loss function computed locally at the target node n. This represents the gradient operator. This represents the model parameters of the target node n in the e-th local iteration.

[0081] In some embodiments, the target node determines the dual variable based on the target difference value, which can be specifically implemented as follows: the target node determines the penalty parameter; the penalty parameter is used to characterize the contribution of the target difference value to the dual variable; the target node determines the dual variable of the current iteration process based on the dual variable, the target difference value and the penalty parameter in the previous iteration process.

[0082] Specifically, the dual variables in this iteration process satisfy the following relationship with the dual variables, target difference values, and penalty parameters in the previous iteration process: ; in, Used to represent the dual variables of node m and node n in the (k+1)th iteration. This is used to represent the dual variable between node m and node n in the k-th iteration, where k is a natural number. Used to represent the penalty parameter (quantifying the severity of the penalty for differences in the models of adjacent nodes). Used to represent the model parameters of node n in the (k+1)th iteration. Used to represent the model parameters of node m during the (k+1)th iteration. Used to represent the target difference value.

[0083] By iteratively updating the dual variables using this formula, we can enhance the feature alignment and semantic mapping of heterogeneous data, achieve deep correlation analysis of multimodal data, and improve the fusion depth of multi-source data.

[0084] To ensure that the deviation between the model parameters of the target node and those of its neighboring nodes is constrained by the dual variable and to avoid parameter divergence, the target node needs to calculate the update direction by combining the target gradient and the consistency constraint with its neighboring nodes. The specific calculation formula is as follows: ; in, This indicates the update direction of the target node n in the e-th local iteration. The dual constraint term represents the difference between adjacent client models. Used to represent the target difference value. This indicates division by the penalty parameter. The cumulative historical model difference parameter after scaling.

[0085] The current update direction not only includes the target gradient, but also superimposes the dual constraint term of the model difference between neighboring nodes, realizing the synergy of convergence acceleration and consistency enhancement under heterogeneous data. This allows the momentum direction to consider both local gradient information and neighbor model consistency, avoiding local optimum shifts caused by data heterogeneity.

[0086] It is the core element of the ADMM mechanism. By constraining the consistency of two adjacent nodes through dual variables and penalty parameters, it ensures that the target model update of the target node does not deviate from the global collaborative goal.

[0087] Each node accumulates historical update directions, and its momentum is calculated using the following formula: ; in, Used to represent the momentum of node n in the (e+1)th iteration. Used to represent the momentum of the target model in the previous iteration. Used to represent momentum parameters (controlling historical momentum weights), where e is a natural number.

[0088] This formula utilizes historical momentum information to uncover deep correlations between data, improves the collaborative efficiency of models in heterogeneous environments, avoids the problem of model divergence due to feature differences, enhances the collaborative efficiency between models, and meets the stringent real-time requirements of industrial scenarios.

[0089] In other words, the momentum in this iteration process satisfies the following relationship with the target gradient, the dual variable, and the momentum of the target model in the previous iteration process: .

[0090] The calculation of momentum depends on the update direction with consistency constraints. The momentum parameter controls the weight of historical momentum and the current update direction, so that the momentum update not only accumulates historical momentum information, but also incorporates the ADMM mechanism to constrain model consistency, achieving global coordination while achieving fast convergence.

[0091] The dual variable captures the differences between models across clients in real time, and the penalty term suppresses model drift caused by heterogeneous data distribution. The momentum term alleviates the problem of large local gradient variance by accumulating historical gradients. This implicit dual-momentum interaction achieves resistance to data heterogeneity without requiring additional hyperparameters, making it highly universal.

[0092] S105. Update the target model based on the momentum of this iteration.

[0093] Specifically, the model update formula can be: ; in, This represents the learning rate, which determines the magnitude of each local model update.

[0094] After completing E local iterations (one round of global iteration), in order to pass the final state of local optimization (including model parameters and momentum state) between rounds, ensure that global iteration training is based on the complete results of local iterations, and continue the historical accumulation of the momentum mechanism to maintain the convergence and consistency of the algorithm, the following settings can be configured: , .in, This represents the model parameters of node n during the (k+1)th round of global iteration. Let represent the model parameters of node n during the E+1th local iteration of the kth global iteration, and momentum is similarly represented.

[0095] After each round of global iteration, the convergence is used to determine the test loss value and accuracy value.

[0096] The iteration is terminated when the global iteration count reaches its maximum value; or, the model is considered to have converged and the iteration is terminated when the loss value is low for several consecutive iterations without a significant downward trend, and the accuracy value increases by no more than 0.1% for several consecutive iterations.

[0097] Each node (such as factory equipment, enterprise servers, or edge nodes) trains model parameters independently locally without uploading raw data, ensuring that sensitive industrial data never leaves the local domain, thus fundamentally eliminating privacy leaks and data compliance risks. By exchanging only model gradients or parameter update information and employing secure aggregation and differential privacy mechanisms, cross-organizational and cross-industry data value sharing is achieved without sacrificing data security. Simultaneously, it effectively breaks down data silos and access permission barriers, enabling semantic-level fusion of multi-source data originally scattered across different industrial systems, supporting cross-domain collaborative analysis and intelligent decision-making.

[0098] By introducing dual variables and penalty terms into the optimization objective, different nodes are forced to maintain consistency constraints in the global parameter space, while allowing local models to retain personalized features, thus achieving a dynamic balance between global sharing and local differences. This approach enables model adaptation at both the feature space and semantic levels, effectively mitigating the distribution drift problem caused by data heterogeneity, thereby supporting deep correlation analysis and knowledge fusion of multimodal and multidimensional industrial data.

[0099] The current parameter update is guided by a weighted historical gradient direction. This mechanism maintains the continuity of the optimization direction while effectively suppressing gradient fluctuations caused by differences in node data distribution or noise, thereby reducing model divergence. Momentum updates not only accelerate the convergence speed of the local model but also enhance the adaptability to non-independent and identically distributed (Non-IID) data. In industrial scenarios, it can significantly shorten the model training cycle, improve real-time response capabilities, and ensure training stability and global consistency in complex network environments.

[0100] An example, such as Figure 4 As shown, the target network is an industry chain composed of upstream, midstream, and downstream enterprises, all of which are nodes. Upstream enterprises possess raw material collection data and production process data, midstream enterprises possess vulnerability detection data and equipment log data, and downstream enterprises possess user information data and market sales data. In a scenario of joint security defense and control among upstream and downstream enterprises in the industry chain, these enterprises need to collaborate to defend against malicious attacks (such as hacker attacks, ransomware attacks, and supply chain injections). However, the security data of each enterprise (such as equipment log data and vulnerability detection data) is sensitive information. Data silos can cause collaborative defense among enterprises to fail. Sensitive data encryption prevents direct sharing, ultimately rendering collaborative defense among upstream, midstream, and downstream enterprises ineffective.

[0101] This application's method treats each enterprise as an independent node, training intrusion detection models based on local security data. This enables collaborative optimization of model parameters across enterprises, achieving "data usable but invisible" throughout the entire industry chain. The decentralized architecture avoids storing all security data on a central server, reducing the risk of centralized data leakage and protecting the privacy and security of raw data from upstream and downstream enterprises. Simultaneously, the consistency constraints of the ADMM mechanism ensure cross-enterprise model feature space alignment, ultimately constructing a collaborative defense system covering the entire industry chain, enabling joint prevention and control of cross-linked threats and intelligent response.

[0102] Another example, such as Figure 5 As shown, the target network is a network composed of factory areas. Factory area A is connected to a temperature sensor through a PLC controller and outputs floating-point data, which is semi-structured data and eventually forms the structured data, the equipment log. Factory area B is also connected to a temperature sensor through a PLC controller, but outputs temperature data that is linearly mapped from the current signal. Its data format is also semi-structured data and eventually forms the structured data, the equipment log. Due to the different brands and models of the equipment from different manufacturers, the output data of the two factory areas are heterogeneous and difficult to merge.

[0103] It can be seen that in the scenario of predictive maintenance of equipment in a cross-plant smart factory, the sensor data, control data, and production data collected exhibit highly heterogeneous characteristics due to significant differences in the production tasks, production materials, and equipment deployed in each plant. This heterogeneous data is distributed both on the local gateway at the edge and stored in a distributed database in the cloud, resulting in insufficient accuracy in predicting equipment failures across plants.

[0104] The method described in this application employs the ADMM mechanism to effectively bridge model differences between different plant areas through consistency constraints, ensuring semantic consistency across plant areas. A decentralized architecture is used to directly exchange model parameters, avoiding format incompatibility issues during cross-plant data transmission. Furthermore, when equipment collects abnormal data, resulting in significant changes in model parameters, the ADMM mechanism iteratively updates the dual variable, automatically increasing the penalty parameter to strengthen the influence of the edge model on cloud parameters, ensuring that fault signals are preferentially transmitted to cross-plant area models.

[0105] Another example, such as Figure 6 As shown, the target network is a network composed of workshops. At the edge server end: Workshop A collects temperature data via a temperature sensor, Workshop B collects vibration data via a vibration sensor, and Workshop C collects smoke data via a smoke sensor. All raw data is uploaded to a cloud data center for processing and results are returned. However, there are two delays in this process: delays caused by network transmission and delays caused by computing power queuing during centralized data processing in the cloud. In other words, cross-workshop equipment fault prediction requires real-time fusion of raw sensor data and historical data from the cloud, but the high latency of centralized cloud processing cannot meet the real-time requirements of cross-workshop equipment industrial control.

[0106] The method described in this application employs a decentralized architecture, enabling direct communication and exchange of model parameters without the need for a central server, thus reducing communication latency and improving the real-time performance of cross-workshop equipment fault prediction. The momentum term accumulates historical momentum, reducing gradient oscillations while accelerating the discovery of deep correlations between vibration data and process parameters, thereby enhancing the ability to identify real-time faults across different workshops.

[0107] Figure 3 The technical solution shown brings at least the following beneficial effects: During model training, each node acquires and references model parameters from neighboring nodes without sharing local training samples. This allows for collaborative improvement of model capabilities while protecting local data privacy. Furthermore, since only model parameters are transmitted between nodes, the amount of data exchanged and the bandwidth requirements are reduced.

[0108] The target loss represents the local data-driven optimization direction of a node, the target difference value represents the globally consistent and coordinated optimization direction from neighboring nodes, and the historical momentum represents the inertial optimization direction of the training process. The target loss, target difference value, and historical momentum together determine the momentum that characterizes the update direction and magnitude of model parameters. In other words, the requirement for consistent collaborative optimization participates in and influences the model optimization trajectory of each node from the outset. This effectively alleviates the problem of model divergence and training oscillation caused by heterogeneous data between different nodes, reconciles the contradiction between a node's own optimization objective and the model state of neighboring nodes, and guides nodes to spontaneously converge towards the network consensus while pursuing their own optimal performance, thereby significantly improving training stability and overall convergence efficiency. Consistent collaborative optimization allows the model to learn the detection capabilities of other nodes in different situations without transmitting raw data between nodes, ultimately resulting in a better-performing model.

[0109] The following describes the model training method for real-time generalized fusion of multi-source heterogeneous industrial data provided in this application from the perspective of overall logic, such as... Figure 7 As shown: First, initialize the model parameters, including initial model parameters, dual variables, learning rate, penalty parameters, and momentum parameters; All nodes execute the k-th round of global iteration in parallel: Receive model parameters from neighboring nodes; Start the e-th round of local iteration: Calculate the target gradient and momentum; Update the target model; Swap the model parameters of adjacent nodes and update the dual variables; Assign the value e = e + 1; Determine if e is less than or equal to E; E is the number of local iterations in a global iteration; if e ≤ E, start the (e+1)th local iteration; if e > E, assign k = k+1. Determine if k is less than or equal to K; K is the maximum number of global iterations; if k ≤ K, start the (k+1)th global iteration; if k > K, output the final model and end the iteration.

[0110] From the perspective of optimization efficiency, the accumulation of historical momentum and update direction under ADMM constraints through the momentum update mechanism not only accelerates the convergence speed of local iteration, but also reduces the frequency of cross-node communication through nested iteration design, thus solving the pain points of large communication overhead and slow convergence of traditional distributed algorithms.

[0111] From the perspective of global consistency, relying on the dual variable and penalty term mechanism of ADMM, by constraining the differences in model parameters of adjacent nodes, it can still ensure that multi-node models converge to the global optimal solution under the decentralized architecture, avoid the parameter divergence problem in heterogeneous data scenarios, and break through the limitation that distributed training is prone to getting trapped in local optima.

[0112] From the perspective of scenario adaptability, decentralized communication mode and privacy protection mechanism are naturally suited to the characteristics of industrial data that are "distributed and privacy-sensitive," enabling cross-domain collaborative learning without centralized data collection. At the same time, the algorithm's compatibility with heterogeneous data allows it to stably support the model training needs of complex industrial scenarios such as intelligent manufacturing and predictive maintenance of equipment.

[0113] This triple guarantee of "efficiency, consistency, and privacy" makes the model training method for real-time general fusion of multi-source heterogeneous industrial data proposed in this application have significant practical value in the field of distributed collaborative learning.

[0114] The foregoing primarily describes the solution provided in this application from a methodological perspective. It is understood that, to achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0115] like Figure 8 The diagram shown is a structural schematic of a model training device 10 for real-time generalized fusion of multi-source heterogeneous industrial data provided in an embodiment of this application. The model training device 10 for real-time generalized fusion of multi-source heterogeneous industrial data includes a processing module 101 and an acquisition module 102.

[0116] The processing module 101 is used to determine the target loss based on the local training samples of the target node and the target model deployed in the target node; the acquisition module 102 is used to acquire the model parameters of the model to be trained in the neighboring nodes of the target node; the processing module 101 is also used to determine the target difference value based on the model parameters of the model to be trained and the model parameters of the target model; the processing module 101 is also used to determine the momentum of the current iteration process based on the target loss, the target difference value and the momentum of the target model in the previous iteration process; the momentum is used to characterize the update direction and magnitude of the model parameters; the processing module 101 is also used to update the target model based on the momentum of the current iteration process.

[0117] Optionally, the processing module 101 is specifically used to: determine the target gradient based on the target loss; the target gradient is used to characterize the adjustment direction of the model parameters that reduce the target loss; determine the dual variable based on the target difference value; the dual variable is used to characterize the degree of difference in model parameters between the target node and its neighboring nodes; and determine the momentum of the current iteration based on the target gradient, the dual variable, and the momentum of the target model in the previous iteration.

[0118] Optionally, the processing module 101 is specifically used to: determine the penalty parameter; the penalty parameter is used to characterize the contribution of the target difference value to the dual variable; and determine the dual variable of the current iteration process based on the dual variable, the target difference value, and the penalty parameter in the previous iteration process.

[0119] Optionally, the dual variable in this iteration process satisfies the following relationship with the dual variable, target difference value, and penalty parameter in the previous iteration process: ;in, Used to represent the dual variables of node m and node n in the (k+1)th iteration. This is used to represent the dual variable between node m and node n in the k-th iteration, where k is a natural number. Used to represent penalty parameters Used to represent the model parameters of node n in the (k+1)th iteration. Used to represent the model parameters of node m during the (k+1)th iteration. Used to represent the target difference value.

[0120] Optionally, the momentum in this iteration process satisfies the following relationship with the target gradient, the dual variable, and the momentum of the target model in the previous iteration process: ;in, Used to represent the momentum of node n in the (e+1)th iteration. Used to represent the target gradient. Used to represent dual variables, Used to represent the momentum of the target model in the previous iteration. Used to represent penalty parameters Used to represent the target difference value. Used to represent the momentum parameter, where e is a natural number.

[0121] This application embodiment can divide the model training device for real-time generalized fusion of multi-source heterogeneous industrial data into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0122] This application also provides an electronic device, including a processor and a memory. The memory stores computer execution instructions, and the processor is connected to the memory. When the electronic device is running, the processor executes the computer execution instructions stored in the memory, so that the electronic device executes any of the industrial multi-source heterogeneous data real-time generalized fusion model training methods provided in the above embodiments.

[0123] The hardware structure of electronic devices can be as follows: Figure 9 As shown, the electronic device includes a processor 201, a memory 202, a communication interface 203, and a bus 204. The memory 202 exists independently of the processor 201. The processor 201, the memory 202, and the communication interface 203 can be connected via the bus 204.

[0124] Processor 201 is the control center of an electronic device. It can be a single processor or a collective term for multiple processing elements. For example, processor 201 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0125] As one embodiment, processor 201 may include one or more CPUs, for example Figure 9 CPU 0 and CPU 1 are shown in the diagram.

[0126] The memory 202 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0127] The memory 202 can be connected to the processor 201 via the bus 204 and is used to store instructions or program code. When the processor 201 calls and executes the instructions or program code stored in the memory 202, it can realize the model training method for real-time general fusion of industrial multi-source heterogeneous data provided in this embodiment of the invention.

[0128] The communication interface 203 is used to connect with other devices via a communication network. The communication network can be Ethernet, a wireless access network, a wireless local area network (WLAN), etc. The communication interface 203 may include a receiving unit for receiving data and a transmitting unit for sending data.

[0129] Bus 204 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0130] In other embodiments, the memory 202 may also be integrated with the processor 201. For example... Figure 10 As shown, the electronic device may include a processor 205 and a communication interface 206, with the processor 205 coupled to the communication interface 206.

[0131] The functions of processor 205 can be referred to the above. Figure 9The processor 201 is described above. Furthermore, the processor 205 also has storage capabilities, which can be found in the above description. Figure 9 The function of memory 202.

[0132] The communication interface 206 is used to provide data to the processor 205. This communication interface 206 can be an internal interface of the electronic device or an external interface of the electronic device (equivalent to...). Figure 9 (Communication interface 203 in the middle).

[0133] It should be pointed out that, Figure 9 (or Figure 10 The structures shown in the document do not constitute a limitation on the electronic device, except... Figure 9 (or Figure 10 In addition to the components shown in the diagram, the electronic device may include more or fewer components than those shown, or combine certain components, or have different component arrangements.

[0134] This application also provides a computer-readable storage medium including computer-executable instructions that, when run on a computer, cause the computer to execute any of the industrial multi-source heterogeneous data real-time generalized fusion model training methods provided in the above embodiments.

[0135] This application also provides a computer program product containing computer execution instructions, which, when run on a computer, causes the computer to execute any of the industrial multi-source heterogeneous data real-time general fusion model training methods provided in the above embodiments.

[0136] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer-executable instructions. When these computer-executable instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method for real-time generalized fusion of multi-source heterogeneous industrial data, characterized in that, Applied to a target node, wherein the target node is any one node in a target network containing multiple nodes; Each of the aforementioned nodes is equipped with a training model designed to perform the same task. The method includes: The target loss is determined based on the local training samples of the target node and the target model deployed in the target node; Obtain the model parameters of the model to be trained in the neighboring nodes of the target node; Based on the model parameters of the model to be trained and the model parameters of the target model, the target difference value is determined; Based on the target loss, the target difference value, and the momentum of the target model in the previous iteration, the momentum of the current iteration is determined; the momentum is used to characterize the update direction and magnitude of the model parameters. The target model is updated based on the momentum generated in this iteration.

2. The method according to claim 1, characterized in that, The determination of momentum for the current iteration based on the target loss, the target difference value, and the momentum of the target model in the previous iteration includes: Based on the target loss, a target gradient is determined; the target gradient is used to characterize the direction of adjustment of the model parameters that reduces the target loss. Based on the target difference value, a dual variable is determined; the dual variable is used to characterize the degree of difference in model parameters between the target node and its neighboring nodes. Based on the target gradient, the dual variable, and the momentum of the target model in the previous iteration, the momentum of the current iteration is determined.

3. The method according to claim 2, characterized in that, The determination of the dual variable based on the target difference value includes: Determine the penalty parameter; the penalty parameter is used to characterize the contribution of the target difference value to the dual variable; Based on the dual variables from the previous iteration, the target difference value, and the penalty parameter, the dual variables for the current iteration are determined.

4. The method according to claim 3, characterized in that, The dual variable in this iteration process satisfies the following relationship with the dual variable in the previous iteration process, the target difference value, and the penalty parameter: ； in, Used to represent the dual variables of node m and node n in the (k+1)th iteration. This is used to represent the dual variable between node m and node n in the k-th iteration, where k is a natural number. Used to represent the penalty parameter. Used to represent the model parameters of node n in the (k+1)th iteration. Used to represent the model parameters of node m during the (k+1)th iteration. Used to represent the target difference value.

5. The method according to claim 2, characterized in that, The momentum in this iteration process satisfies the following relationship with the target gradient, the dual variable, and the momentum of the target model in the previous iteration process: ； in, Used to represent the momentum of node n in the (e+1)th iteration. Used to represent the target gradient Used to represent the dual variable, This is used to represent the momentum of the target model in the previous iteration. Used to represent penalty parameters Used to represent the target difference value. Used to represent the momentum parameter, where e is a natural number.

6. The method according to claim 1, characterized in that, The step of obtaining the model parameters of the model to be trained in the neighboring nodes of the target node includes: The model parameters of the model to be trained in the neighboring nodes of the target node are obtained according to a preset period, wherein the preset period includes a preset number of iterations.

7. A model training device for real-time generalized fusion of multi-source heterogeneous industrial data, characterized in that, include: The processing module is used to determine the target loss based on the training samples local to the target node and the target model deployed in the target node; The acquisition module is used to acquire the model parameters of the model to be trained in the neighboring nodes of the target node; The processing module is further configured to determine a target difference value based on the model parameters of the model to be trained and the model parameters of the target model. The processing module is further configured to determine the momentum of the current iteration based on the target loss, the target difference value, and the momentum of the target model in the previous iteration; the momentum is used to characterize the update direction and magnitude of the model parameters. The processing module is also used to update the target model based on the momentum of the current iteration.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor; The memory and the processor are coupled; The memory is used to store computer program code, which includes computer instructions; Wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes computer execution instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-6.

Images