Dual-mode dual-channel data processing method and system based on edge calculation and federated learning

Abstract

The invention provides a dual-mode dual-channel data processing method and system based on edge calculation and federated learning, and relates to the technical field of electric power. According to the specific implementation scheme, in the updating process of a global model for processing industrial data by a central computing node, weights are dynamically allocated according to the data quality, computing power and data acquisition frequency of each edge computing node; the nodes with higher data quality, higher computing power and higher data acquisition frequency contribute more to global model parameters, and local model parameters of each edge computing node are updated by using the global model parameters. By adopting the method, the accuracy and the stability of the global model and the local model of each edge computing node on industrial data processing are improved.

Description

Technical Field

[0001] The present invention relates to the field of big data processing technology, and in particular to a dual-mode dual-channel data processing method and system based on edge computing and federated learning. Background Art

[0002] In large-scale production environments, centralized data collection systems are usually relied upon. The system achieves centralized data management by transmitting all data to a central server for unified processing. However, in actual production environments, with the rapid increase in data volume and the continuous improvement of real-time requirements, traditional centralized data collection systems have exposed a series of problems.

[0003] First, the central server can easily become a system bottleneck under high-load scenarios, especially when performing stress testing in a large-scale production environment. The delay in data transmission and processing is very prominent, making it difficult for the system response time to meet the requirements of minute-level data collection. In addition, with the popularization of edge devices and IoT technology, more and more terminal devices are distributed at the production site, and the centralized system cannot fully utilize the computing power of these edge devices, resulting in a waste of resources.

[0004] In order to deal with these problems, some technologies have proposed solutions to improve data processing efficiency through distributed computing or edge computing. Among them, distributed computing usually relies on complex node coordination mechanisms, which is difficult to implement in practical applications. Although edge computing technology has alleviated the load pressure of central servers to a certain extent, it still faces great challenges in data privacy protection.

[0005] At the same time, although some federated learning technologies provide certain solutions for distributed data collaborative training, their application in large-scale production environments still faces many difficulties. In some technologies, the update strategy of the federated learning model is usually unable to dynamically allocate weights according to the data quality and computing power of each node, resulting in poor global optimization of the model. Summary of the invention

[0006] The present invention provides a dual-mode dual-channel data processing method, system, electronic device and storage medium based on edge computing and federated learning, which can solve at least one of the above technical problems.

[0007] According to one aspect of the present invention, there is provided an industrial equipment data processing method, comprising: The central computing node receives the industrial equipment data collected locally by each edge computing node among the multiple edge computing nodes, and locally obtains the local model parameters trained based on the industrial equipment data and the gradient information corresponding to the local model parameters; The central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; The central computing node determines the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; The central computing node determines the dynamic allocation weight of each edge computing node based on the data collection quality, computing power and data collection frequency of each edge computing node for the industrial equipment data; The central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights; The central computing node sends the global model parameters to each of the edge computing nodes, so that each of the edge computing nodes updates the local model parameters of each edge computing node based on the global model parameters.

[0008] According to another aspect of the present invention, a dual-mode dual-channel data processing system based on edge computing and federated learning is provided, comprising a central computing node and a plurality of edge computing nodes, wherein the central computing node comprises: An edge data receiving module, used to receive the industrial equipment data collected locally by each edge computing node, and local model parameters obtained by training the industrial equipment data locally and gradient information corresponding to the local model parameters; A data quality determination module, used to determine the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; A computing capacity determination module, used to determine the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; A dynamic weight determination module, used to determine the dynamic allocation weight of each edge computing node based on the data collection quality, computing power and data collection frequency of each edge computing node for the industrial equipment data; A global parameter determination module, used to determine global model parameters based on local model parameters of each edge computing node and gradient information corresponding to the local model parameters, and dynamically allocated weights; The local parameter updating module is used to send the global model parameters to each edge computing node, so that each edge computing node updates the local model parameters of each edge computing node based on the global model parameters.

[0009] According to another aspect of the present invention, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any dual-mode dual-channel data processing method based on edge computing and federated learning in the embodiments of the present invention.

[0010] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any dual-mode dual-channel data processing method based on edge computing and federated learning according to the embodiments of the present invention.

[0011] By adopting the technical solution of the present invention, a central computing node receives industrial equipment data collected locally by each edge computing node among multiple edge computing nodes, and locally obtains local model parameters trained based on the industrial equipment data and gradient information corresponding to the local model parameters; the central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; the central computing node determines the computing power of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; the central computing node determines the dynamic allocation weight of each edge computing node based on the data collection quality, computing power and data collection frequency for industrial equipment data of each edge computing node; the central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weight; the central computing node sends the global model parameters to each edge computing node, so that each edge computing node updates the local model parameters of each edge computing node based on the global model parameters. Therefore, in the process of updating the global model used by the central computing node to process industrial data, weights are dynamically allocated according to the data quality, computing power and data collection frequency of each edge computing node, so that nodes with higher data quality, stronger computing power and higher data collection frequency contribute more to the global model parameters, and the global model parameters are used to update the local model parameters of each edge computing node, thereby improving the accuracy and stability of the global model and the local models of each edge computing node in processing industrial data.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention. Figure 1 It is a flow chart of a dual-mode dual-channel data processing method based on edge computing and federated learning according to an embodiment of the present invention; Figure 2 It is a structural block diagram of a dual-mode dual-channel data processing system based on edge computing and federated learning according to an embodiment of the present invention; Figure 3 The block diagram is a block diagram of an electronic device for implementing the method according to the embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0015] Figure 1 It is a flowchart of a dual-mode dual-channel data processing method based on edge computing and federated learning according to an embodiment of the present invention.

[0016] like Figure 1 As shown, the dual-mode dual-channel data processing method based on edge computing and federated learning may include: S110, the central computing node receives industrial equipment data collected locally by each edge computing node among multiple edge computing nodes, and locally obtains local model parameters and gradient information corresponding to the local model parameters based on the industrial equipment data training; S120, the central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; S130, the central computing node determines the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization, and bandwidth resource utilization of each edge computing node; S140, the central computing node determines a dynamic allocation weight of each edge computing node based on the data collection quality, computing power, and data collection frequency of each edge computing node for industrial equipment data; S150, the central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights; S160: The central computing node sends the global model parameters to each edge computing node, so that each edge computing node updates the local model parameters of each edge computing node based on the global model parameters.

[0017] For example, in a large-scale production environment, a large amount of industrial equipment data may be generated, for example, the operating status data of multiple devices, covering multimodal information such as temperature, vibration, and pressure. Different edge computing nodes use the operating status data of different devices.

[0018] For example, in a large-scale production environment, the above-mentioned edge computing nodes can be obtained by deploying edge computing devices for each data collection node. The edge computing device includes a computing module and a storage module. The computing module is responsible for performing local data processing tasks, and the storage module is used to store the collected raw data and pre-processed data.

[0019] Exemplarily, when the data collection node completes data collection, the edge computing device starts a local data preprocessing operation. The preprocessing operation may include data cleaning, format conversion, and data compression.

[0020] Exemplarily, the preprocessing operation may be expressed by the following formula, as follows: in, Represents the preprocessed industrial equipment data, Represents raw industrial equipment data without preprocessing, represents the data cleaning function, represents the data compression function, Represents the intermediate data after the format conversion of the original industrial equipment data.

[0021] Exemplarily, the edge computing device dynamically adjusts its data transmission strategy according to the preprocessing results. For example, it chooses to transmit data to the central computing node immediately, or chooses to cache data locally for a period of time, and then transmit the data to the central computing node after reaching a preset data storage amount. In this way, the transmission delay can be optimized when the transmission amount is large.

[0022] Exemplarily, when using data, the edge computing node can enable a dual-mode dual-channel acquisition mechanism. The mechanism includes a synchronous acquisition mode and an asynchronous acquisition mode. Among them, the synchronous acquisition mode is used for data acquisition in high-load scenarios, and the asynchronous acquisition mode is used for data acquisition in low-load scenarios. The edge computing node can dynamically switch the data acquisition mode according to the real-time load status.

[0023] For example, in the synchronous collection mode, the edge computing node transmits the collected industrial equipment data to the central computing node through a high-speed communication channel, thereby minimizing the data collection delay in high-load scenarios.

[0024] Exemplarily, the calculation formula of the data transmission rate in the synchronous acquisition mode is as follows: in, Indicates the data transmission rate of industrial equipment data in synchronous acquisition mode, Indicates the amount of industrial equipment data collected under high-load scenarios, Indicates the minimum allowed transmission time.

[0025] For example, in the asynchronous acquisition mode, the edge computing node selects an appropriate time to transmit data through a low-speed communication channel according to the actual data volume and load conditions, thereby reducing system resource usage in low-load scenarios.

[0026] Exemplarily, the calculation formula of the data transmission rate in asynchronous mode is as follows: in, Indicates the data transmission rate of industrial equipment data in asynchronous mode, Indicates the amount of industrial equipment data collected in low-load scenarios, Indicates variable transmission time to dynamically adjust to varying load demands.

[0027] According to the above example, the edge computing node can dynamically switch between synchronous acquisition mode and asynchronous acquisition mode according to its load status, thereby optimizing the resource utilization and data transmission efficiency of the edge computing node.

[0028] Exemplarily, the central computing node determines the data collection quality of the edge computing node based on the data cleanliness, completeness, and consistency of the industrial equipment data collected locally by the edge computing node.

[0029] Exemplarily, the calculation formula for the data collection quality of the edge computing node can be as follows: in, represents the data collection quality of the kth edge computing node, Indicates the number of samples of industrial equipment data collected locally by the edge computing node, represents the effective data volume of the i-th sample in the industrial equipment data collected locally by the edge computing node, is the total data volume of the ith sample.

[0030] Exemplarily, the central computing node may perform a weighted average of the CPU utilization, memory utilization, and bandwidth resource utilization of each edge computing node to determine the computing capacity of each edge computing node.

[0031] Exemplarily, the calculation formula of the computing power of the edge computing node can be as follows: in, represents the computing power of the kth edge computing node, represents the CPU utilization of the kth edge computing node, represents the memory utilization of the kth edge computing node, Indicates the bandwidth resource utilization of the kth edge computing node.

[0032] Exemplarily, the central computing node performs weighted summation of the data collection quality, computing power, and data collection frequency for industrial equipment data of each edge computing node to obtain a dynamic allocation weight for each edge computing node.

[0033] Exemplarily, the calculation formula for the dynamic allocation weight of the edge computing node is as follows: in, represents the dynamic allocation weight of the kth edge computing node, represents the data collection quality of the kth edge computing node, represents the computing power of the kth edge computing node, represents the data collection frequency of the kth edge computing node for industrial equipment data, Represents the total number of edge computing nodes, , and They represent the weight coefficients of data collection quality, computing power and data collection frequency respectively.

[0034] In one embodiment, in the above step S150, the central computing node determines the calculation formula of the global model parameter based on the local model parameter of the edge computing node and the gradient information corresponding to the local model parameter, and the dynamic allocation weight as follows: in, represents the updated global model parameters, represents the global model parameters before updating, represents the learning rate, Indicates the number of nodes of the above multiple edge computing nodes, represents the dynamic allocation weight of the kth edge node, represents the gradient information of the kth edge node, represents the computing resource utilization of the kth edge node, Indicates the upper limit of computing resource utilization.

[0035] According to the above implementation mode, in the process of updating the global model of industrial data processed by the central computing node, weights are dynamically allocated according to the data quality, computing power and data collection frequency of each edge computing node, so that nodes with higher data quality, stronger computing power and higher data collection frequency contribute more to the global model parameters, which can improve the accuracy and stability of the global model in processing industrial data.

[0036] After the central computing node calculates the global model parameters, it sends the global model parameters to each edge node through the dual-channel edge communication mechanism.

[0037] In one implementation, after receiving the optimized global model parameters, each edge computing node may use the global model parameters to update its local model parameters.

[0038] Exemplarily, the calculation formula for the edge computing node to update its local model parameters using the global model parameters is as follows: in, represents the updated local model parameters, represents the local model parameters before updating, The updated global model parameters, Represents the weight coefficient.

[0039] Exemplarily, the edge computing node adjusts the data collection frequency and the communication link selection strategy according to the updated local model parameters.

[0040] Exemplarily, the adjustment formula for the data collection frequency of the edge computing node is as follows: in, represents the updated data collection frequency of the kth edge computing node, represents the data collection frequency of the kth edge computing node before updating, represents the latest accuracy of the local model of the kth edge computing node, represents the latest accuracy of the global model of the central computing node, Represents the weight coefficient.

[0041] Understandably, when the accuracy of the local model is high, it means that the data of the current edge computing node has performed well in training the local model, so the data collection frequency can be appropriately reduced to save resources. If the accuracy of the local model is low, the data collection frequency can be increased to obtain more data to improve model performance.

[0042] Understandably, a high-precision local model may mean that the current edge computing node already has enough data, so the frequency of data upload can be reduced or a low-bandwidth communication link can be selected to reduce resource consumption. For a low-precision local model, the current edge node may select a higher-bandwidth communication link to speed up data updates.

[0043] According to the above example, the accuracy of the local model directly affects the data collection frequency and communication link selection strategy of the current edge computing node. Through this mechanism, the data collection frequency and communication link selection strategy are dynamically adjusted under different accuracy conditions, which can optimize the resource usage of edge computing while meeting the global model training requirements.

[0044] According to any of the above-mentioned implementation modes, in the process of updating the global model for processing industrial data by the central computing node, weights are dynamically allocated according to the data quality, computing power and data collection frequency of each edge computing node, so that nodes with higher data quality, stronger computing power and higher data collection frequency contribute more to the global model parameters, and the global model parameters are used to update the local model parameters of each edge computing node, thereby improving the accuracy and stability of the global model and the local model of each edge computing node in processing industrial data.

[0045] Before the above step S110, after each edge computing node obtains the industrial equipment data, it can train the local model according to the following example. Among them, the local model is used to predict the corresponding industrial equipment status, data collection strategy and data transmission strategy based on the industrial equipment data collected locally. At the same time, the corresponding edge communication channel can also be selected for data transmission based on the data volume of the industrial equipment data and other information. The specific example is as follows.

[0046] In one embodiment, the above method also includes: when the industrial equipment data collected by the edge computing node meets the preset conditions, based on the industrial equipment data, training the local model of the edge computing node to obtain the local model parameters of the edge computing node and the gradient information corresponding to the local model parameters; the edge computing node determines the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time requirements of data transmission; the edge computing node sends the industrial equipment data, local model parameters, and at least one of the gradient information corresponding to the local model parameters to the central computing node through the edge communication channel at the current moment. Time series data at the current moment.

[0047] It is understandable that each edge computing node can execute the above examples.

[0048] Exemplarily, the edge computing node constructs a local training data set based on the preprocessed industrial equipment data, and trains the local model using the local data set to obtain local model parameters and gradient information corresponding to the local model parameters.

[0049] Exemplarily, the goal of local model training is to minimize the loss function with a regularization term. The loss function is as follows: in, represents the loss function of local model training, represents the i-th training sample ( , ), are local model parameters, represents the number of samples in the local training dataset, represents the regularization coefficient, is the regularization term.

[0050] For example, after the edge computing node completes the local model training, the edge computing node converts the local model parameters obtained by the local training into , and the gradient information corresponding to the local model parameters It is sent to the central computing node through the above edge communication channel for updating the global model parameters.

[0051] Exemplarily, the gradient information corresponding to the local model parameters may be as follows: in, represents the i-th training sample ( , ) The partial derivative of the loss function corresponding to, represents the derivative of the local model output with respect to the local model parameters, is the derivative of the regularization term with respect to the local model parameters.

[0052] It can be understood that while calculating the local model parameters, the following steps can also be performed moment by moment: the edge computing node determines the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time requirements of data transmission; the edge computing node sends the industrial equipment data, local model parameters, and at least one of the gradient information corresponding to the local model parameters at the current moment to the central computing node through the edge communication channel at the current moment. Time series data.

[0053] According to the above implementation, the edge computing node can train local model parameters based on the industrial equipment data collected locally, and select a suitable edge communication channel to transmit the industrial equipment data, local model parameters, and at least one of the gradient information corresponding to the local model parameters at the current moment to the central computing node.

[0054] The following introduces the edge computing node's selection strategy for the edge communication channel. The details are as follows.

[0055] In one embodiment, the edge computing node determines the calculation formula of the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time data transmission requirements: in, represents the edge communication channel at the current moment, represents the weighted weight function, Indicates the real-time data transmission requirements of industrial equipment data. Indicates the current amount of industrial equipment data. represents the selection probability of selecting the first type of communication link at the current moment, represents the bandwidth of the first type of communication link, Represents the bandwidth of the second type of communication link, and the transmission rate of the first type of communication link is greater than the transmission speed of the second type of communication link.

[0056] In this example, based on the data volume and real-time requirements, a suitable link is selected for data transmission. For example, the choice between different communication links (such as high-speed links and low-speed links). For another example, under the current data volume and real-time requirements, the edge computing node should determine whether to use a high-speed communication link or a low-speed communication link first.

[0057] In one embodiment, the edge computing node determines the edge communication channel at the current moment based on the current data volume and real-time data transmission requirements of the industrial equipment data collected locally, including: the edge computing node determines the selection probability of selecting the first type of communication link at the current moment according to the data volume and real-time data transmission requirements of the industrial equipment data at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment; the edge computing node determines the edge communication channel at the current moment based on the selection probability of selecting the first type of communication link at the current moment, and the current data volume and real-time data transmission requirements of the industrial equipment data.

[0058] In one embodiment, the edge computing node determines the calculation formula for selecting the first type of communication link at the current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at the previous moment, and the bandwidth utilization of the edge communication channel at the previous moment: in, represents the selection probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, Indicates the amount of industrial equipment data at the last moment. Indicates the real-time data transmission requirements of industrial equipment data. Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment. and Represents the weight coefficient.

[0059] Exemplarily, the adjustment of the communication link type means dynamically changing the link selection tendency according to the current data volume, real-time requirements and bandwidth conditions. For example, when the data volume increases or the real-time requirements increase, the system may tend to choose a high-speed link. When the data volume decreases or the real-time requirements decrease, the system may tend to choose a low-speed link to save resources.

[0060] Exemplarily, for the data volume and data transmission real-time requirements, if the data volume is large or the data transmission real-time requirements are small, the value of the selection probability of selecting the first type of communication link at the current moment will increase, which means that the edge computing node is more inclined to select the current high-bandwidth link type.

[0061] Exemplarily, for the current bandwidth utilization, if the bandwidth utilization of the current link is close to the bandwidth upper limit, the denominator of the above formula increases, resulting in a decrease in the selection probability, thereby prompting the edge computing node to switch to another link type with a higher probability in the next time step.

[0062] For example, the probability of selecting a communication link reflects the degree of trust that the edge computing node has in the current link type, and this probability is automatically adjusted as the amount of data, real-time requirements, and bandwidth conditions change. By dynamically adjusting the probability of selecting a communication link, the link type can be flexibly switched at different time steps, thereby optimizing resource utilization while meeting real-time requirements.

[0063] It can be understood that the probability of selecting a communication link at the next time step is actually the system's intention to use the current link type. When conditions change, the selection probability will also be adjusted, prompting the system to switch link types to adapt to new communication needs and network conditions. This dynamic adjustment mechanism enables adaptive selection of link types, thereby optimizing transmission performance and allocation of network resources.

[0064] In this example, based on the communication link currently selected, the selection probability of the link is further dynamically adjusted. The selection strategy of the communication link is dynamically adjusted according to the data volume, the real-time requirements of data transmission, and the changes in the actual bandwidth and bandwidth upper limit of the current communication link. By adjusting the selection probability, the link selection is made more adaptive, and the resource utilization and transmission efficiency of the link are optimized.

[0065] In one embodiment, the above method also includes: when the edge computing node detects that data transmission abnormalities occur in the edge communication channel, based on the congestion level of the edge communication channel and the corresponding congestion threshold, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel, determining an activation strategy for the backup communication channel; the edge computing node replaces the edge communication channel based on the activation strategy for the backup communication channel, so that the edge computing node sends industrial equipment data, local model parameters, and gradient information corresponding to the local model parameters to the central computing node through the replaced edge communication channel.

[0066] Exemplarily, the data transmission anomaly may be communication link congestion or link interruption.

[0067] In one embodiment, based on the congestion level of the edge communication channel and the corresponding congestion threshold, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel, a calculation formula for determining the activation strategy of the backup communication channel is: in, Indicates the activation strategy of the backup communication channel. is a step function, Indicates the congestion level of the edge communication channel, Indicates the real-time data transmission requirements of industrial equipment data. represents the congestion threshold, represents the bandwidth of the backup communication channel, Indicates the bandwidth of the edge communication channel.

[0068] According to the above implementation, during data transmission, if a communication link is congested or interrupted, a backup communication link with a bandwidth higher than a preset value may be enabled.

[0069] The embodiment of the present invention deploys edge computing devices at data collection nodes to transfer some data processing tasks to edge devices, which greatly reduces the load on the central server and achieves real-time data collection at the minute level. The embodiment of the present invention reduces the amount of transmitted data through local data preprocessing operations, optimizes the delay performance of data collection, greatly improves the response speed of the system, and can effectively reduce transmission delays.

[0070] In the embodiment of the present invention, weights are dynamically allocated according to the data quality, computing power and data collection frequency of each edge node during the global model update process of federated learning. Compared with the weight processing of each node in the prior art, the dynamic allocation of weights enables nodes with higher data quality and stronger computing power to contribute more to the global model, thereby improving the accuracy and stability of the global model and solving the problem of poor global model effect caused by inconsistent data quality and computing power between nodes in the existing federated learning system.

[0071] The embodiment of the present invention designs a dual-mode dual-channel data acquisition and transmission mechanism, adopts a synchronous acquisition mode and an asynchronous acquisition mode respectively, uses a high-speed communication link in a high-load scenario to ensure the real-time performance of data acquisition, and uses a low-speed communication link in a low-load scenario to reduce the occupation of system resources. The data acquisition frequency and the transmission channel are dynamically adjusted according to the actual load to optimize the utilization of system resources, thereby ensuring the stable operation of the system under different production pressures. Compared with the existing single data acquisition method, the present invention effectively improves the data transmission efficiency and ensures the real-time performance and stability of data acquisition.

[0072] Figure 2 It is a structural block diagram of a dual-mode dual-channel data processing system based on edge computing and federated learning according to an embodiment of the present disclosure.

[0073] like Figure 2 As shown, the dual-mode dual-channel data processing system based on edge computing and federated learning includes a central computing node 210 and multiple edge computing nodes 220, wherein the central computing node includes: The edge data receiving module 211 is used to receive the industrial equipment data collected locally by each edge computing node, and the local model parameters obtained by training the industrial equipment data locally and the gradient information corresponding to the local model parameters; A data quality determination module 212, configured to determine the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; A computing capacity determination module 213, configured to determine the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization, and bandwidth resource utilization of each edge computing node; A dynamic weight determination module 214, configured to determine a dynamic allocation weight of each edge computing node based on the data collection quality, computing power, and data collection frequency of each edge computing node for the industrial equipment data; A global parameter determination module 215, configured to determine global model parameters based on local model parameters of each edge computing node and gradient information corresponding to the local model parameters, as well as dynamically allocated weights; The local parameter updating module 216 is used to send the global model parameters to each of the edge computing nodes, so that each of the edge computing nodes updates the local model parameters of each edge computing node based on the global model parameters.

[0074] In one implementation, the central computing node determines the calculation formula of the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights as follows: in, represents the updated global model parameters, represents the global model parameters before updating, represents the learning rate, represents the number of edge computing nodes, represents the dynamic allocation weight of the kth edge node, represents the gradient information of the kth edge node, represents the computing resource utilization of the kth edge node, Indicates the upper limit of computing resource utilization.

[0075] In one embodiment, the edge computing node 220 includes: A local model training module is used to train the local model of the edge computing node based on the industrial equipment data when the industrial equipment data collected by the edge computing node meets the preset conditions, and obtain the local model parameters of the edge computing node and the gradient information corresponding to the local model parameters; A communication channel determination module, used to determine the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time requirement of data transmission; The data transmission module is used to send the time series data of at least one of the industrial equipment data, the local model parameters and the gradient information corresponding to the local model parameters at the current moment to the central computing node through the edge communication channel at the current moment.

[0076] In one embodiment, the edge computing node determines the calculation formula of the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time data transmission requirements: in, represents the edge communication channel at the current moment, represents the weighted weight function, Indicates the real-time data transmission requirements of the industrial equipment data, represents the current data volume of the industrial equipment data, represents the selection probability of selecting the first type of communication link at the current moment, represents the bandwidth of the first type of communication link, Represents the bandwidth of the second type of communication link, and the transmission rate of the first type of communication link is greater than the transmission speed of the second type of communication link.

[0077] In one implementation, the communication channel determination module is specifically configured to: Determine the selection probability of selecting the first type of communication link at the current moment according to the data volume and data transmission real-time requirement of the industrial equipment data at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment; Based on the selection probability of selecting the first type of communication link at the current moment, and the current data volume of the industrial equipment data and the real-time data transmission requirement, the edge communication channel at the current moment is determined.

[0078] In one embodiment, the edge computing node determines the calculation formula for selecting the first type of communication link at the current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at the previous moment, and the bandwidth utilization of the edge communication channel at the previous moment: in, represents the probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, represents the data volume of the industrial equipment data at the last moment, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment.

[0079] In one embodiment, the edge computing node 220 further includes: A backup strategy determination module, configured to determine, when detecting data transmission anomalies in the edge communication channel, an activation strategy of the backup communication channel based on the congestion level of the edge communication channel and the corresponding congestion threshold, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel; The communication channel switching module is used to replace the edge communication channel based on the activation strategy of the backup communication channel, so that the edge computing node sends the industrial equipment data, the local model parameters and the gradient information corresponding to the local model parameters to the central computing node through the replaced edge communication channel.

[0080] For the description of specific functions and examples of each module and submodule in the system of the embodiment of the present invention, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0081] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0082] According to an embodiment of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0083] Figure 3A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0084] like Figure 3 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0085] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0086] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a dual-mode dual-channel data processing method based on edge computing and federated learning. For example, in some embodiments, the dual-mode dual-channel data processing method based on edge computing and federated learning may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the dual-mode dual-channel data processing method based on edge computing and federated learning described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (eg, by means of firmware) to perform a dual-mode dual-channel data processing method based on edge computing and federated learning.

[0087] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0091] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0092] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0093] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0094] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dual-mode dual-channel data processing method based on edge computing and federated learning, characterized in that: include: The central computing node receives the industrial equipment data collected locally by each edge computing node among the multiple edge computing nodes, and locally obtains the local model parameters trained based on the industrial equipment data and the gradient information corresponding to the local model parameters; The central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; The central computing node determines the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; The central computing node determines the dynamic allocation weight of each edge computing node based on the data collection quality, computing power and data collection frequency of each edge computing node for the industrial equipment data; The central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights; The central computing node sends the global model parameters to each of the edge computing nodes, so that each of the edge computing nodes updates the local model parameters of each edge computing node based on the global model parameters.

2. The method according to claim 1, characterized in that The central computing node determines the calculation formula of the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights as follows: in, represents the updated global model parameters, represents the global model parameters before updating, represents the learning rate, represents the number of nodes of the plurality of edge computing nodes, represents the dynamic allocation weight of the kth edge node, represents the gradient information of the kth edge node, represents the computing resource utilization of the kth edge node, Indicates the upper limit of computing resource utilization.

3. The method according to claim 1, characterized in that Also includes: When the industrial equipment data collected by the edge computing node meets the preset conditions, the local model of the edge computing node is trained based on the industrial equipment data to obtain the local model parameters of the edge computing node and the gradient information corresponding to the local model parameters; The edge computing node determines the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time requirement of data transmission; The edge computing node sends the time series data of at least one of the industrial equipment data, the local model parameters, and the gradient information corresponding to the local model parameters at the current moment to the central computing node through the edge communication channel at the current moment.

4. The method according to claim 3, characterized in that The edge computing node determines the calculation formula of the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time data transmission requirements: in, represents the edge communication channel at the current moment, represents the weighted weight function, Indicates the real-time data transmission requirements of the industrial equipment data, represents the current data volume of the industrial equipment data, represents the selection probability of selecting the first type of communication link at the current moment, represents the bandwidth of the first type of communication link, Represents the bandwidth of the second type of communication link, and the transmission rate of the first type of communication link is greater than the transmission speed of the second type of communication link.

5. The method according to claim 3, characterized in that: The edge computing node determines the edge communication channel at the current moment based on the current moment data volume of the industrial equipment data collected locally and the real-time requirement of data transmission, including: The edge computing node determines the selection probability of selecting the first type of communication link at the current moment according to the data volume and data transmission real-time requirement of the industrial equipment data at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment; The edge computing node determines the edge communication channel at the current moment based on the selection probability of selecting the first type of communication link at the current moment, and the current data volume of the industrial equipment data and the real-time data transmission requirements.

6. The method according to claim 5, characterized in that The edge computing node determines the calculation formula for selecting the first type of communication link at the current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment: in, represents the probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, represents the data volume of the industrial equipment data at the last moment, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment. and Represents the weight coefficient.

7. The method according to claim 3, characterized in that Also includes: The edge computing node determines, when detecting that data transmission abnormality occurs in the edge communication channel, an activation strategy of the backup communication channel based on the congestion degree of the edge communication channel and the corresponding congestion threshold, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel; The edge computing node replaces the edge communication channel based on the activation strategy of the backup communication channel, so that the edge computing node sends the industrial equipment data, the local model parameters and the gradient information corresponding to the local model parameters to the central computing node through the replaced edge communication channel.

8. A dual-mode dual-channel data processing system based on edge computing and federated learning, characterized in that: The system comprises a central computing node and a plurality of edge computing nodes, wherein the central computing node comprises: An edge data receiving module, used to receive the industrial equipment data collected locally by each edge computing node, and local model parameters obtained by training the industrial equipment data locally and gradient information corresponding to the local model parameters; A data quality determination module, used to determine the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; A computing capacity determination module, used to determine the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; A dynamic weight determination module, used to determine the dynamic allocation weight of each edge computing node based on the data collection quality, computing power and data collection frequency of each edge computing node for the industrial equipment data; A global parameter determination module, used to determine global model parameters based on local model parameters of each edge computing node and gradient information corresponding to the local model parameters, and dynamically allocated weights; The local parameter updating module is used to send the global model parameters to each edge computing node, so that each edge computing node updates the local model parameters of each edge computing node based on the global model parameters.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images