Dual-mode dual-channel data processing method and system based on edge computing and federated learning

Abstract

The present invention provides a dual-mode, dual-channel data processing method and system based on edge computing and federated learning, relating to the field of power technology. A specific implementation scheme is as follows: During the update process of the global model used by the central computing node to process industrial data, weights are dynamically assigned to each edge computing node based on the data quality, computing power, and data acquisition frequency. This ensures that nodes with higher data quality, stronger computing power, and higher data acquisition frequency contribute more to the global model parameters, and the global model parameters are used to update the local model parameters of each edge computing node. The present invention improves the accuracy and stability of industrial data processing by the global model and the local models of each edge computing node.

Description

Technical Field

[0001] The present invention relates to the field of big data processing technology, and in particular to a dual-mode dual-channel data processing method and system based on edge computing and federated learning. Background Art

[0002] Large-scale production environments typically rely on centralized data collection systems, which centrally manage data by transmitting all data to a central server for unified processing. However, in real-world production environments, with the rapid increase in data volume and increasing real-time requirements, traditional centralized data collection systems have exposed a series of problems.

[0003] First, central servers can easily become system bottlenecks under high load. This is especially true during stress testing in large-scale production environments, where data transmission and processing delays are particularly prominent, making it difficult for system response times to meet minute-level data collection requirements. Furthermore, with the proliferation of edge devices and IoT technology, an increasing number of terminal devices are distributed across production sites. Centralized systems are unable to fully utilize the computing power of these edge devices, resulting in wasted resources.

[0004] To address these issues, some technologies have proposed improving data processing efficiency through distributed computing or edge computing. Distributed computing typically relies on complex node coordination mechanisms, making it challenging to implement in practice. While edge computing technology has alleviated the load on central servers to some extent, it still presents significant challenges in protecting data privacy.

[0005] Meanwhile, while some federated learning technologies offer solutions for distributed data collaborative training, their application in large-scale production environments still faces numerous challenges. In some technologies, the update strategies used in federated learning models often fail to dynamically assign weights based on the data quality and computing power of each node, resulting in poor global optimization. Summary of the Invention

[0006] The present invention provides a dual-mode dual-channel data processing method, system, electronic device and storage medium based on edge computing and federated learning, which can solve at least one of the above technical problems.

[0007] According to one aspect of the present invention, there is provided a method for processing industrial equipment data, comprising:

[0008] The central computing node receives industrial equipment data collected locally by each edge computing node among multiple edge computing nodes, and locally trains local model parameters and gradient information corresponding to the local model parameters based on the industrial equipment data;

[0009] The central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node;

[0010] The central computing node determines the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node;

[0011] The central computing node determines a dynamic allocation weight of each edge computing node based on the data collection quality, computing power, and data collection frequency of each edge computing node for the industrial equipment data;

[0012] The central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights;

[0013] The central computing node sends the global model parameters to each of the edge computing nodes, so that each of the edge computing nodes updates its local model parameters based on the global model parameters.

[0014] According to another aspect of the present invention, a dual-mode dual-channel data processing system based on edge computing and federated learning is provided, comprising a central computing node and multiple edge computing nodes, wherein the central computing node comprises:

[0015] An edge data receiving module, configured to receive the industrial equipment data collected locally by each of the edge computing nodes, and local model parameters trained locally based on the industrial equipment data and gradient information corresponding to the local model parameters;

[0016] a data quality determination module, configured to determine the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node;

[0017] A computing capacity determination module, configured to determine the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization, and bandwidth resource utilization of each edge computing node;

[0018] a dynamic weight determination module, configured to determine a dynamic allocation weight for each of the edge computing nodes based on the data collection quality, computing power, and data collection frequency of the industrial equipment data of each of the edge computing nodes;

[0019] A global parameter determination module, configured to determine global model parameters based on local model parameters of each edge computing node and gradient information corresponding to the local model parameters, as well as dynamically assigned weights;

[0020] The local parameter updating module is used to send the global model parameters to each edge computing node, so that each edge computing node updates its local model parameters based on the global model parameters.

[0021] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any dual-mode dual-channel data processing method based on edge computing and federated learning in the embodiments of the present invention.

[0025] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any dual-mode dual-channel data processing method based on edge computing and federated learning according to the embodiments of the present invention.

[0026] By adopting the technical solution of the present invention, the central computing node receives the industrial equipment data collected locally by each edge computing node among multiple edge computing nodes, and locally obtains local model parameters and gradient information corresponding to the local model parameters based on the industrial equipment data training; the central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; the central computing node determines the computing power of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; the central computing node determines the dynamic allocation weight of each edge computing node based on the data collection quality, computing power and data collection frequency for industrial equipment data of each edge computing node; the central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weight; the central computing node sends the global model parameters to each edge computing node, so that each edge computing node updates the local model parameters of each edge computing node based on the global model parameters. Therefore, in the process of updating the global model used by the central computing node to process industrial data, weights are dynamically allocated based on the data quality, computing power and data acquisition frequency of each edge computing node, so that nodes with higher data quality, stronger computing power and higher data acquisition frequency contribute more to the global model parameters, and the global model parameters are used to update the local model parameters of each edge computing node, thereby improving the accuracy and stability of the global model and the local model of each edge computing node in processing industrial data.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention.

[0029] Figure 1 This is a flow chart of a dual-mode dual-channel data processing method based on edge computing and federated learning according to an embodiment of the present invention;

[0030] Figure 2 This is a structural block diagram of a dual-mode, dual-channel data processing system based on edge computing and federated learning according to an embodiment of the present invention;

[0031] Figure 3 is a block diagram of an electronic device for implementing the method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, and various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] Figure 1 This is a flowchart of a dual-mode dual-channel data processing method based on edge computing and federated learning according to an embodiment of the present invention.

[0034] like Figure 1 As shown, the dual-mode dual-channel data processing method based on edge computing and federated learning may include:

[0035] S110, the central computing node receives industrial equipment data collected locally by each edge computing node among multiple edge computing nodes, and locally trains local model parameters and gradient information corresponding to the local model parameters based on the industrial equipment data;

[0036] S120, the central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node;

[0037] S130, the central computing node determines the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization, and bandwidth resource utilization of each edge computing node;

[0038] S140, the central computing node determines a dynamic allocation weight for each edge computing node based on the data collection quality, computing capability, and data collection frequency of each edge computing node for industrial equipment data;

[0039] S150, the central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamically allocated weights;

[0040] S160: The central computing node sends the global model parameters to each edge computing node, so that each edge computing node updates its local model parameters based on the global model parameters.

[0041] For example, in a large-scale production environment, a large amount of industrial equipment data can be generated. For example, the operating status data of multiple devices covers multimodal information such as temperature, vibration, and pressure. Different edge computing nodes use the operating status data of different devices.

[0042] For example, in a large-scale production environment, the aforementioned edge computing nodes can be obtained by deploying edge computing devices for each data collection node. The edge computing devices include a computing module and a storage module. The computing module is responsible for performing local data processing tasks, and the storage module is used to store the collected raw data and pre-processed data.

[0043] For example, after the data collection node completes data collection, the edge computing device initiates local data preprocessing operations, which may include data cleaning, format conversion, and data compression.

[0044] For example, the preprocessing operation can be expressed by the following formula:

[0045]

[0046] in, Represents pre-processed industrial equipment data, Represents raw industrial equipment data without preprocessing, Represents the data cleaning function, represents the data compression function, Represents intermediate data after format conversion of original industrial equipment data.

[0047] For example, the edge computing device dynamically adjusts its data transmission strategy based on the preprocessing results. For example, it may choose to immediately transmit data to the central computing node, or to cache the data locally for a period of time, and then transmit the data to the central computing node after a preset data storage capacity is reached. This can optimize transmission latency when the data volume is large.

[0048] For example, when consuming data, edge computing nodes can enable a dual-mode, dual-channel acquisition mechanism. This mechanism includes synchronous and asynchronous acquisition modes. Synchronous acquisition mode is used for data acquisition in high-load scenarios, while asynchronous acquisition mode is used for data acquisition in low-load scenarios. Edge computing nodes can dynamically switch data acquisition modes based on real-time load conditions.

[0049] For example, in synchronous collection mode, edge computing nodes transmit collected industrial equipment data to central computing nodes via high-speed communication channels, thereby minimizing data collection delays in high-load scenarios.

[0050] For example, the data transmission rate in the synchronous acquisition mode is calculated as follows:

[0051]

[0052] in, Indicates the data transmission rate of industrial equipment data in synchronous acquisition mode, Indicates the amount of industrial equipment data collected under high-load scenarios, Indicates the minimum allowed transmission time.

[0053] For example, in the asynchronous acquisition mode, the edge computing node selects an appropriate time to transmit data through a low-speed communication channel based on the actual data volume and load conditions, thereby reducing system resource usage in low-load scenarios.

[0054] For example, the calculation formula for the data transmission rate in asynchronous mode is as follows:

[0055]

[0056] in, Indicates the data transmission rate of industrial equipment data in asynchronous mode, Indicates the amount of industrial equipment data collected in low-load scenarios, Indicates variable transmission time to dynamically adjust to different load demands.

[0057] According to the above example, the edge computing node can dynamically switch between synchronous acquisition mode and asynchronous acquisition mode according to its load status, thereby optimizing the resource utilization and data transmission efficiency of the edge computing node.

[0058] Exemplarily, the central computing node determines the data collection quality of the edge computing node based on the data cleanliness, completeness, and consistency of the industrial equipment data collected locally by the edge computing node.

[0059] For example, the calculation formula for the data collection quality of the edge computing node can be as follows:

[0060]

[0061] in, represents the data collection quality of the kth edge computing node, Indicates the number of samples of industrial equipment data collected locally by the edge computing node, represents the effective data volume of the i-th sample in the industrial equipment data collected locally by the edge computing node, is the total data volume of the i-th sample.

[0062] For example, the central computing node may perform a weighted average of the CPU utilization, memory utilization, and bandwidth resource utilization of each edge computing node to determine the computing capacity of each edge computing node.

[0063] For example, the calculation formula of the computing power of the edge computing node can be as follows:

[0064]

[0065] in, represents the computing power of the kth edge computing node, represents the CPU utilization of the kth edge computing node, represents the memory utilization of the kth edge computing node, Indicates the bandwidth resource utilization of the kth edge computing node.

[0066] Exemplarily, the central computing node performs weighted summation on the data collection quality, computing power, and data collection frequency for industrial equipment data of each edge computing node to obtain a dynamic allocation weight for each edge computing node.

[0067] For example, the calculation formula for the dynamic allocation weight of the edge computing node is as follows:

[0068]

[0069] in, represents the dynamic allocation weight of the kth edge computing node, represents the data collection quality of the kth edge computing node, represents the computing power of the kth edge computing node, represents the data collection frequency of the kth edge computing node for industrial equipment data, Indicates the total number of edge computing nodes, 、 and They represent the weight coefficients of data collection quality, computing power and data collection frequency respectively.

[0070] In one embodiment, in the above step S150, the central computing node determines the calculation formula of the global model parameters based on the local model parameters of the edge computing nodes and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights as follows:

[0071]

[0072] in, represents the updated global model parameters, represents the global model parameters before updating, represents the learning rate, Indicates the number of the above-mentioned edge computing nodes, represents the dynamic allocation weight of the kth edge node, represents the gradient information of the kth edge node, represents the computing resource utilization of the kth edge node, Indicates the upper limit of computing resource utilization.

[0073] According to the above implementation, in the process of updating the global model of industrial data processed by the central computing node, weights are dynamically allocated based on the data quality, computing power and data acquisition frequency of each edge computing node, so that nodes with higher data quality, stronger computing power and higher data acquisition frequency contribute more to the global model parameters, which can improve the accuracy and stability of the global model in processing industrial data.

[0074] After the central computing node calculates the global model parameters, it sends the global model parameters to each edge node through the dual-channel edge communication mechanism.

[0075] In one embodiment, after receiving the optimized global model parameters, each edge computing node may use the global model parameters to update its local model parameters.

[0076] Exemplarily, the calculation formula for the edge computing node to update its local model parameters using the global model parameters is as follows:

[0077]

[0078] in, represents the updated local model parameters, represents the local model parameters before updating, The updated global model parameters, Represents the weight coefficient.

[0079] Exemplarily, the edge computing node adjusts the data collection frequency and communication link selection strategy according to the updated local model parameters.

[0080] For example, the adjustment formula for the data collection frequency of the edge computing node is as follows:

[0081]

[0082] in, represents the updated data collection frequency of the kth edge computing node, represents the data collection frequency of the kth edge computing node before updating, represents the latest accuracy of the local model of the kth edge computing node, represents the latest accuracy of the global model of the central computing node, Represents the weight coefficient.

[0083] Understandably, when the local model's accuracy is high, it indicates that the data from the current edge computing node has performed well in training the local model, so the data collection frequency can be appropriately reduced to save resources. If the local model's accuracy is low, the data collection frequency can be increased to obtain more data to improve model performance.

[0084] Understandably, a high-precision local model may mean that the current edge computing node already has sufficient data, so it can reduce the frequency of data uploads or select a low-bandwidth communication link to reduce resource consumption. For a low-precision local model, the current edge node may select a higher-bandwidth communication link to speed up data updates.

[0085] In the above example, the accuracy of the local model directly affects the data collection frequency and communication link selection strategy of the current edge computing node. Through this mechanism, dynamically adjusting the data collection frequency and communication link selection strategy under different accuracy conditions can optimize edge computing resource utilization while meeting the global model training requirements.

[0086] According to any of the above-mentioned embodiments, in the process of updating the global model of industrial data processed by the central computing node, weights are dynamically allocated according to the data quality, computing power and data acquisition frequency of each edge computing node, so that nodes with higher data quality, stronger computing power and higher data acquisition frequency contribute more to the global model parameters, and the global model parameters are used to update the local model parameters of each edge computing node, thereby improving the accuracy and stability of the global model and the local model of each edge computing node in processing industrial data.

[0087] Before step S110 above, after each edge computing node obtains industrial equipment data, it can train a local model according to the following example. The local model is used to predict the corresponding industrial equipment status, data collection strategy, and data transmission strategy based on the locally collected industrial equipment data. Furthermore, the corresponding edge communication channel can be selected for data transmission based on information such as the volume of industrial equipment data. A specific example is shown below.

[0088] In one embodiment, the above method also includes: when the industrial equipment data collected by the edge computing node meets the preset conditions, based on the industrial equipment data, training the local model of the edge computing node to obtain the local model parameters of the edge computing node and the gradient information corresponding to the local model parameters; the edge computing node determines the edge communication channel at the current moment based on the current data volume of the locally collected industrial equipment data and the real-time transmission requirements of the data; the edge computing node sends the time series data of at least one of the industrial equipment data, local model parameters and the gradient information corresponding to the local model parameters at the current moment to the central computing node through the edge communication channel at the current moment.

[0089] It is understandable that each edge computing node can execute the above examples.

[0090] For example, the edge computing node constructs a local training dataset based on the preprocessed industrial equipment data, trains a local model using the local dataset, and obtains local model parameters and gradient information corresponding to the local model parameters.

[0091] For example, the goal of local model training is to minimize the loss function with a regularization term. The loss function is as follows:

[0092]

[0093] in, represents the loss function for local model training, represents the i-th training sample ( , ), are local model parameters, represents the number of samples in the local training dataset, represents the regularization coefficient, is the regularization term.

[0094] For example, after the edge computing node completes the local model training, the edge computing node uses the local model parameters obtained by local training , and the gradient information corresponding to the local model parameters It is sent to the central computing node through the above-mentioned edge communication channel for updating the global model parameters.

[0095] For example, the gradient information corresponding to the local model parameters may be as follows:

[0096]

[0097] in, represents the i-th training sample ( , ) The partial derivative of the loss function corresponding to represents the derivative of the local model output with respect to the local model parameters, is the derivative of the regularization term with respect to the local model parameters.

[0098] It can be understood that while calculating the local model parameters, the following steps can also be performed moment by moment: the edge computing node determines the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time requirements of data transmission; the edge computing node sends the industrial equipment data, local model parameters and at least one of the gradient information corresponding to the local model parameters at the current moment to the central computing node through the edge communication channel at the current moment.

[0099] According to the above embodiment, the edge computing node can train local model parameters based on the industrial equipment data collected locally, and select a suitable edge communication channel to transmit the industrial equipment data, local model parameters and at least one of the gradient information corresponding to the local model parameters at the current moment to the central computing node.

[0100] The following describes the edge computing node's selection strategy for the edge communication channel.

[0101] In one embodiment, the edge computing node determines the edge communication channel at the current moment based on the current data volume of the locally collected industrial equipment data and the real-time data transmission requirements. The calculation formula is:

[0102]

[0103] in, Represents the edge communication channel at the current moment, represents the weighted weight function, Indicates the real-time data transmission requirements of industrial equipment data, Indicates the current amount of industrial equipment data. represents the probability of selecting the first type of communication link at the current moment, represents the bandwidth of the first type of communication link, Represents the bandwidth of the second type of communication link, and the transmission rate of the first type of communication link is greater than the transmission speed of the second type of communication link.

[0104] In this example, the appropriate link for data transmission is selected based on the data volume and real-time requirements. For example, the choice is made between different communication links (such as high-speed links and low-speed links). Another example is whether the edge computing node should prioritize high-speed or low-speed communication links based on the current data volume and real-time requirements.

[0105] In one embodiment, the edge computing node determines the edge communication channel at the current moment based on the current data volume and real-time data transmission requirements of the locally collected industrial equipment data, including: the edge computing node determines the selection probability of selecting the first type of communication link at the current moment based on the data volume and real-time data transmission requirements of the industrial equipment data at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment; the edge computing node determines the edge communication channel at the current moment based on the selection probability of selecting the first type of communication link at the current moment, and the current data volume and real-time data transmission requirements of the industrial equipment data.

[0106] In one embodiment, the edge computing node determines the probability of selecting the first type of communication link at the current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at the previous moment, as well as the bandwidth utilization of the edge communication channel at the previous moment, using the following calculation formula:

[0107]

[0108] in, represents the probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, Indicates the amount of industrial equipment data at the last moment. Indicates the real-time data transmission requirements of industrial equipment data, Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment, and Represents the weight coefficient.

[0109] For example, adjusting the communication link type means dynamically changing the link selection preference based on the current data volume, real-time requirements, and bandwidth conditions. For example, as data volume increases or real-time requirements increase, the system may prefer a high-speed link. As data volume decreases or real-time requirements decrease, the system may prefer a low-speed link to conserve resources.

[0110] For example, with respect to data volume and data transmission real-time requirements, if the data volume is large or the data transmission real-time requirements are small, the value of the selection probability of selecting the first type of communication link at the current moment will increase, which means that the edge computing node is more inclined to select the current high-bandwidth link type.

[0111] For example, for the current bandwidth utilization, if the bandwidth utilization of the current link is close to the bandwidth upper limit, the denominator of the above formula increases, resulting in a decrease in the selection probability, thereby prompting the edge computing node to have a higher probability of switching to another link type in the next time step.

[0112] For example, the probability of selecting a communication link reflects the edge computing node's confidence in the current link type. This probability is automatically adjusted as data volume, real-time requirements, and bandwidth conditions change. By dynamically adjusting the probability of selecting a communication link, link types can be flexibly switched at different time steps, optimizing resource utilization while still meeting real-time requirements.

[0113] Understandably, the probability of selecting a communication link at the next time step actually reflects the system's intention to use the current link type. As conditions change, the selection probability adjusts accordingly, prompting the system to switch link types to adapt to new communication needs and network conditions. This dynamic adjustment mechanism enables adaptive link selection, thereby optimizing transmission performance and network resource allocation.

[0114] In this example, the link selection probability is dynamically adjusted based on the currently selected communication link. This strategy is dynamically adjusted based on data volume, real-time data transmission requirements, and changes in the actual bandwidth and bandwidth cap of the current communication link. Adjusting the selection probability makes link selection more adaptive, optimizing link resource utilization and transmission efficiency.

[0115] In one embodiment, the above method also includes: when the edge computing node detects that a data transmission anomaly occurs in the edge communication channel, the edge computing node determines an activation strategy for the backup communication channel based on the congestion level of the edge communication channel and the corresponding congestion threshold, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel; the edge computing node replaces the edge communication channel based on the activation strategy for the backup communication channel, so that the edge computing node sends industrial equipment data, local model parameters, and gradient information corresponding to the local model parameters to the central computing node through the replaced edge communication channel.

[0116] Illustratively, the data transmission anomaly may be communication link congestion or link interruption.

[0117] In one embodiment, based on the congestion level and corresponding congestion threshold of the edge communication channel, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel, a calculation formula for determining the activation strategy of the backup communication channel is:

[0118]

[0119] in, Indicates the activation strategy of the backup communication channel. is a step function, Indicates the congestion level of the edge communication channel, Indicates the real-time data transmission requirements of industrial equipment data, represents the congestion threshold, Indicates the bandwidth of the backup communication channel, Indicates the bandwidth of the edge communication channel.

[0120] According to the above embodiment, during data transmission, if a communication link is congested or interrupted, a backup communication link with a bandwidth higher than a preset value may be enabled.

[0121] This embodiment of the present invention deploys edge computing devices at data collection nodes, offloading some data processing tasks to edge devices. This significantly reduces the load on central servers and enables real-time data collection within minutes. This embodiment of the present invention also reduces the amount of data transmitted through local data preprocessing, optimizes data collection latency, significantly improves system response speed, and effectively reduces transmission delays.

[0122] In an embodiment of the present invention, weights are dynamically allocated based on the data quality, computing power, and data acquisition frequency of each edge node during the global model update process of federated learning. Compared with the weight processing of each node in the prior art, by dynamically allocating weights, nodes with higher data quality and stronger computing power contribute more to the global model, thereby improving the accuracy and stability of the global model and solving the problem of poor global model effect caused by inconsistent data quality and computing power between nodes in the existing federated learning system.

[0123] The embodiment of the present invention designs a dual-mode dual-channel data acquisition and transmission mechanism, adopts synchronous acquisition mode and asynchronous acquisition mode respectively, uses high-speed communication links in high-load scenarios to ensure the real-time performance of data acquisition, and uses low-speed communication links in low-load scenarios to reduce the occupation of system resources. The data acquisition frequency and transmission channel are dynamically adjusted according to the actual load to optimize the utilization of system resources, ensuring the stable operation of the system under different production pressures. Compared with the existing single data acquisition method, the present invention effectively improves the data transmission efficiency and ensures the real-time performance and stability of data acquisition.

[0124] Figure 2 This is a structural block diagram of a dual-mode, dual-channel data processing system based on edge computing and federated learning according to an embodiment of the present disclosure.

[0125] like Figure 2 As shown, the dual-mode dual-channel data processing system based on edge computing and federated learning includes a central computing node 210 and multiple edge computing nodes 220, wherein the central computing node includes:

[0126] The edge data receiving module 211 is used to receive the industrial equipment data collected locally by each edge computing node, and the local model parameters and gradient information corresponding to the local model parameters obtained by training based on the industrial equipment data.

[0127] a data quality determination module 212 for determining the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node;

[0128] A computing capacity determination module 213 is configured to determine the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization, and bandwidth resource utilization of each edge computing node;

[0129] A dynamic weight determination module 214 is configured to determine a dynamic allocation weight for each edge computing node based on the data collection quality, computing power, and data collection frequency of each edge computing node for the industrial equipment data;

[0130] A global parameter determination module 215 is configured to determine global model parameters based on the local model parameters of each edge computing node, gradient information corresponding to the local model parameters, and dynamically assigned weights;

[0131] The local parameter updating module 216 is configured to send the global model parameters to each edge computing node, so that each edge computing node updates its local model parameters based on the global model parameters.

[0132] In one embodiment, the central computing node determines the calculation formula of the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights as follows:

[0133]

[0134] in, represents the updated global model parameters, represents the global model parameters before updating, represents the learning rate, represents the number of edge computing nodes. represents the dynamic allocation weight of the kth edge node, represents the gradient information of the kth edge node, represents the computing resource utilization of the kth edge node, Indicates the upper limit of computing resource utilization.

[0135] In one embodiment, the edge computing node 220 includes:

[0136] A local model training module is used to train the local model of the edge computing node based on the industrial equipment data collected by the edge computing node when the industrial equipment data meets the preset conditions, and obtain the local model parameters of the edge computing node and the gradient information corresponding to the local model parameters;

[0137] A communication channel determination module, configured to determine an edge communication channel at a current moment based on the current amount of locally collected industrial equipment data and the real-time requirements of data transmission;

[0138] The data transmission module is used to send the time series data of at least one of the industrial equipment data, the local model parameters and the gradient information corresponding to the local model parameters at the current moment to the central computing node through the edge communication channel at the current moment.

[0139] In one embodiment, the edge computing node determines the edge communication channel at the current moment based on the current data volume of the industrial equipment data collected locally and the real-time data transmission requirements as follows:

[0140]

[0141] in, Represents the edge communication channel at the current moment, represents the weighted weight function, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the current data volume of the industrial equipment data, represents the probability of selecting the first type of communication link at the current moment, represents the bandwidth of the first type of communication link, Represents the bandwidth of the second type of communication link, and the transmission rate of the first type of communication link is greater than the transmission speed of the second type of communication link.

[0142] In one embodiment, the communication channel determination module is specifically configured to:

[0143] Determining a probability of selecting the first type of communication link at a current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at a previous moment, and the bandwidth utilization of the edge communication channel at a previous moment;

[0144] Based on the selection probability of selecting the first type of communication link at the current moment, and the current data volume and data transmission real-time requirements of the industrial equipment data at the current moment, the edge communication channel at the current moment is determined.

[0145] In one embodiment, the edge computing node determines the probability of selecting the first type of communication link at the current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at the previous moment, as well as the bandwidth utilization of the edge communication channel at the previous moment, using the following calculation formula:

[0146]

[0147] in, represents the probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, Indicates the amount of data of the industrial equipment data at the last moment, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment.

[0148] In one embodiment, the edge computing node 220 further includes:

[0149] a backup strategy determination module, configured to, upon detecting a data transmission anomaly on the edge communication channel, determine an activation strategy for the backup communication channel based on a congestion level and a corresponding congestion threshold of the edge communication channel, a bandwidth of the backup communication channel, and the bandwidth of the edge communication channel;

[0150] The communication channel switching module is used to replace the edge communication channel based on the activation policy of the backup communication channel, so that the edge computing node sends the industrial equipment data, the local model parameters and the gradient information corresponding to the local model parameters to the central computing node through the replaced edge communication channel.

[0151] For the description of specific functions and examples of each module and submodule in the system of the embodiment of the present invention, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0152] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0153] According to an embodiment of the present invention, the present invention further provides an electronic device and a readable storage medium.

[0154] Figure 3A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0155] like Figure 3 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. Computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0156] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0157] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the dual-mode, dual-channel data processing method based on edge computing and federated learning. For example, in some embodiments, the dual-mode, dual-channel data processing method based on edge computing and federated learning can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the dual-mode, dual-channel data processing method based on edge computing and federated learning described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (for example, by means of firmware) to perform a dual-mode dual-channel data processing method based on edge computing and federated learning.

[0158] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0163] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0164] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved. This is not limited herein.

[0165] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A dual-mode dual-channel data processing method based on edge computing and federated learning, characterized in that: include: The edge computing node determines the edge communication channel at the current moment between the first type of communication link and the second type of communication link based on the current data volume and data transmission real-time requirements of the industrial equipment data collected locally, including: the edge computing node determines the selection probability of selecting the first type of communication link at the current moment based on the data volume and data transmission real-time requirements of the industrial equipment data at the previous moment, the selection probability of selecting the first type of communication link at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment; the edge computing node determines the edge communication channel at the current moment between the first type of communication link and the second type of communication link based on the selection probability of selecting the first type of communication link at the current moment, and the current data volume and data transmission real-time requirements of the industrial equipment data; wherein the transmission rate of the first type of communication link is greater than the transmission speed of the second type of communication link; The calculation formula for determining the selection probability of selecting the first type of communication link at the current moment is: in, represents the probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, Indicates the amount of data of the industrial equipment data at the last moment, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment, and represents the weight coefficient; The edge computing node sends, to the central computing node through the edge communication channel at the current moment, time series data of at least one of the industrial equipment data, the local model parameters, and the gradient information corresponding to the local model parameters at the current moment; The central computing node receives the industrial equipment data collected locally by each of the edge computing nodes, and locally trains the local model parameters obtained based on the industrial equipment data and the gradient information corresponding to the local model parameters; The central computing node determines the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; The central computing node determines the computing capacity of each edge computing node based on the central processing unit utilization, memory utilization and bandwidth resource utilization of each edge computing node; The central computing node determines a dynamic allocation weight of each edge computing node based on the data collection quality, computing power, and data collection frequency of each edge computing node for the industrial equipment data; The central computing node determines the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights; The central computing node sends the global model parameters to each of the edge computing nodes, so that each of the edge computing nodes updates its local model parameters based on the global model parameters.

2. The method according to claim 1, characterized in that The central computing node determines the calculation formula of the global model parameters based on the local model parameters of each edge computing node and the gradient information corresponding to the local model parameters, as well as the dynamic allocation weights as follows: in, represents the updated global model parameters, represents the global model parameters before updating, represents the learning rate, represents the number of edge computing nodes. represents the dynamic allocation weight of the kth edge node, represents the gradient information of the kth edge node, represents the computing resource utilization of the kth edge node, Indicates the upper limit of computing resource utilization.

3. The method according to claim 1, characterized in that Also includes: When the industrial equipment data collected by the edge computing node meets preset conditions, the local model of the edge computing node is trained based on the industrial equipment data to obtain the local model parameters of the edge computing node and the gradient information corresponding to the local model parameters.

4. The method according to claim 3, characterized in that The edge computing node determines the edge communication channel at the current moment based on the current data volume of the locally collected industrial equipment data and the real-time data transmission requirements in the first type of communication link and the second type of communication link. The calculation formula is: in, Represents the edge communication channel at the current moment, represents the weighted weight function, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the current data volume of the industrial equipment data, represents the probability of selecting the first type of communication link at the current moment, represents the bandwidth of the first type of communication link, Indicates the bandwidth of the second type of communication link.

5. The method according to claim 3, characterized in that Also includes: The edge computing node, when detecting that data transmission abnormality occurs in the edge communication channel, determines an activation strategy for the backup communication channel based on the congestion level and the corresponding congestion threshold of the edge communication channel, the bandwidth of the backup communication channel, and the bandwidth of the edge communication channel; The edge computing node replaces the edge communication channel based on the activation policy of the backup communication channel, so that the edge computing node sends the industrial equipment data, the local model parameters and the gradient information corresponding to the local model parameters to the central computing node through the replaced edge communication channel.

6. A dual-mode dual-channel data processing system based on edge computing and federated learning, characterized in that: Including central computing nodes and multiple edge computing nodes, The edge computing nodes include: A communication channel determination module is used to determine the edge communication channel at the current moment in the first type of communication link and the second type of communication link based on the current data volume of the locally collected industrial equipment data and the real-time data transmission requirements; a data transmission module, configured to send, to the central computing node through the edge communication channel at the current moment, time series data of at least one of the industrial equipment data, the local model parameters, and the gradient information corresponding to the local model parameters at the current moment; Among them, the communication channel determination module is specifically used to determine the selection probability of selecting the first type of communication link at the current moment based on the data volume of the industrial equipment data at the previous moment and the real-time transmission requirement of data transmission, the selection probability of selecting the first type of communication link at the previous moment, and the bandwidth utilization rate of the edge communication channel at the previous moment; based on the selection probability of selecting the first type of communication link at the current moment, and the data volume of the industrial equipment data at the current moment and the real-time transmission requirement of data transmission, determine the edge communication channel at the current moment between the first type of communication link and the second type of communication link; The calculation formula for determining the selection probability of selecting the first type of communication link at the current moment is: in, represents the probability of selecting the first type of communication link at the current moment, represents the selection probability of selecting the first type of communication link at the last moment, Indicates the amount of data of the industrial equipment data at the last moment, Indicates the real-time data transmission requirements of the industrial equipment data, Indicates the bandwidth utilization of the edge communication channel at the previous moment, Indicates the actual bandwidth of the edge communication channel at the previous moment, Indicates the bandwidth upper limit of the edge communication channel at the previous moment, and represents the weight coefficient; The central computing node includes: An edge data receiving module, configured to receive the industrial equipment data collected locally by each of the edge computing nodes, and local model parameters trained locally based on the industrial equipment data and gradient information corresponding to the local model parameters; a data quality determination module, configured to determine the data collection quality of each edge computing node based on the industrial equipment data of each edge computing node; A computing capacity determination module, configured to determine the computing capacity of each edge computing node based on the central processor utilization, memory utilization, and bandwidth resource utilization of each edge computing node; a dynamic weight determination module, configured to determine a dynamic allocation weight for each of the edge computing nodes based on the data collection quality, computing power, and data collection frequency of the industrial equipment data of each of the edge computing nodes; A global parameter determination module, configured to determine global model parameters based on local model parameters of each edge computing node and gradient information corresponding to the local model parameters, as well as dynamically assigned weights; The local parameter updating module is used to send the global model parameters to each edge computing node, so that each edge computing node updates its local model parameters based on the global model parameters.

7. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images