An edge-computing-oriented lightweight federated graph autoencoder training method

Abstract

The application discloses a lightweight federated graph autoencoder training method for edge computing, belongs to the field of distributed machine learning and graph neural network, and comprises the following steps: in view of the problems of sparse edge-side graph data, limited communication bandwidth and unstable model training, local graph data representation is enhanced by extracting and fusing high-order triangular motif structures on the client side; a residual sparse compression method with key layer protection and error accumulation mechanism is adopted to greatly reduce the communication data volume and ensure the effectiveness of gradient updating; and a multi-stage scheduling strategy containing warm-up training, periodic full synchronization and learning rate decay is combined to inhibit model drift and ensure stable convergence. The application can effectively reduce communication overhead and improve the training efficiency and node clustering accuracy of the model in the edge environment on the premise of protecting data privacy.

Description

Technical Field

[0001] This invention belongs to the field of distributed machine learning and graph neural networks, and in particular relates to a lightweight federated graph autoencoder training method for edge computing. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and edge computing, massive amounts of graph-structured data are generated at edge devices, such as sensor network topologies and social relationship graphs. To collaboratively utilize this dispersed data while protecting data privacy, federated graph learning technology has emerged. This technology allows multiple participants to jointly train a graph neural network model without sharing the original graph data, becoming an important research direction in the field of privacy-preserving machine learning.

[0003] However, existing federated graph learning methods still face significant challenges in real-world edge computing scenarios. First, the local subgraphs held by individual edge devices are often sparse and homogeneous, making it difficult for models based on first-order adjacency relationships to capture complex high-order topological features, resulting in insufficient discriminative power of the learned node representations. Second, edge network bandwidth resources are limited, and frequent full-model parameter interactions during federated training generate significant communication overhead, failing to meet the practical deployment requirements of low latency and low energy consumption. Furthermore, methods such as gradient compression used to reduce communication costs often crudely discard some information, easily leading to the loss of key gradients and error accumulation, causing problems such as unstable model convergence and performance degradation, making it difficult to achieve a good balance between compressed communication and ensuring model accuracy and training stability. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a lightweight federated graph autoencoder training method for edge computing, comprising:

[0005] Based on local graph data from edge clients, structure-enhanced graph data is constructed by extracting and fusing high-order topology structures;

[0006] Unsupervised training is performed based on the structure enhancement graph data to obtain the gradient update amount of the local model parameters;

[0007] Perform residual sparse compression processing with key layer protection on the gradient update amount to generate sparse gradient information to be uploaded, and accumulate and save the gradient information discarded in this compression process as historical residuals.

[0008] The sparse gradient information is sent to the cloud server so that the cloud server can aggregate information from multiple edge clients and update the global model parameters.

[0009] According to the preset multi-stage scheduling strategy, the training, compression, transmission and aggregation processes are executed iteratively until the model training reaches the preset convergence condition.

[0010] Optionally, the step of constructing the structure-enhanced graph data by extracting and fusing higher-order topological structures includes:

[0011] Based on the adjacency matrix of the local graph data, a square matrix representing the second-order path is obtained by multiplying the adjacency matrix by itself.

[0012] The square matrix is ​​multiplied element-wise with the original adjacency matrix to obtain the motif matrix representing the triangular relationship structure between nodes.

[0013] The motif matrix is ​​smoothed and normalized to obtain the motif weight matrix;

[0014] The motif weight matrix and the original adjacency matrix are weighted and superimposed according to an adjustable fusion coefficient to generate the final structure enhancement graph data.

[0015] Optionally, the smoothing and normalization of the phantom matrix specifically includes:

[0016] First, a smoothing operation is performed on each element value in the motif matrix, which is then incremented by one and the logarithm is taken.

[0017] Then, the values ​​of each element in the smoothed matrix are divided by its global maximum value to complete the normalization.

[0018] Optionally, performing residual sparse compression with key layer protection on the gradient update amount includes:

[0019] Read the historical residual memory left over from the previous training round stored locally;

[0020] The initial gradient update amount calculated in the current round is added to the historical residual memory to obtain the cumulative gradient update amount;

[0021] Identify the parameters belonging to a preset key layer in the accumulated gradient update amount, wherein the key layer includes at least the model's output mapping layer and all bias term parameters;

[0022] The identified key layer parameters are fully retained, their complete values ​​are directly added to the sparse gradient information to be uploaded, and their corresponding historical residual memory is cleared.

[0023] For the cumulative gradient update amount of non-critical layers, according to the preset compression ratio, only the first part of the parameter values ​​with the largest absolute value is retained and sparse gradient information is added.

[0024] The parameter values ​​that were not selected and retained in the non-critical layers are saved as new historical residual memories for compensation in the next training round.

[0025] Optionally, the multi-stage scheduling strategy includes:

[0026] During the initial warm-up phase, gradient compression is disabled, and a full transfer of all parameters is performed.

[0027] During the regular training phase following the warm-up phase, full parameter synchronization rounds are periodically inserted to correct accumulated errors.

[0028] In regular rounds of non-full synchronization, gradient compression is enabled, and the compression ratio is gradually increased to the target value as the training progresses.

[0029] Throughout the training process, the model's learning rate is gradually reduced according to a predetermined schedule.

[0030] Optionally, the compression ratio is gradually increased as the training progresses, and the increase is based on the ratio of the current training round to the total training rounds, linearly increasing from the initial value to the target compression ratio.

[0031] Optionally, the unsupervised training based on the structure-enhanced graph data is implemented through a graph autoencoder model, including:

[0032] The input augmented graph structure and node features are encoded using a graph convolutional network to obtain low-dimensional embedding vectors for the nodes.

[0033] The original graph connectivity is reconstructed by performing inner product operations between the embedded vectors and transforming them using an activation function.

[0034] The parameters of the graph autoencoder model are updated by minimizing the difference loss between the reconstructed graph and the original graph.

[0035] Optionally, the cloud server performs aggregation in the following way:

[0036] Assign appropriate weights to each edge client based on the size of its local graph data.

[0037] The weights are used to perform a weighted average of the sparse gradient information received from each client, and the result of the weighted average is used to update the global model.

[0038] On the other hand, the present invention also provides an electronic device including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0039] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0040] Compared with the prior art, the present invention has the following advantages and technical effects:

[0041] This invention enhances local graph data through higher-order structures, effectively improving the feature representation capabilities of sparse data at the edge and improving the accuracy of downstream tasks such as node clustering. The residual sparse compression mechanism with key layer protection and error compensation significantly reduces the amount of communication data between the client and server while ensuring unbiased gradient updates and guaranteeing model performance. Combined with a multi-stage dynamic scheduling strategy, it effectively alleviates model drift caused by accumulated sparse communication, enabling the model to converge stably even in low-bandwidth environments. The entire method achieves efficient federated collaborative training while strictly adhering to the principle of "data remains stationary while the model moves," strengthening the privacy and security of sensitive industrial data during transmission at the algorithmic level. This provides a feasible technical solution for privacy protection and efficient collaborative analysis of graph data in edge computing environments. Attached Figure Description

[0042] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0043] Figure 1 This is an overall flowchart of a lightweight federated graph autoencoder training method for edge computing according to an embodiment of the present invention.

[0044] Figure 2 This is a schematic diagram illustrating the principle of local map data high-order structure enhancement (M3 phantom fusion) in an embodiment of the present invention;

[0045] Figure 3 This is a logic block diagram of a residual sparse compression mechanism with critical layer protection according to an embodiment of the present invention.

[0046] Figure 4 This is a schematic diagram of the system architecture applied to a distributed industrial Internet of Things scenario according to an embodiment of the present invention. Detailed Implementation

[0047] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0048] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0049] Example 1

[0050] This embodiment provides a lightweight federated graph autoencoder training method for edge computing, including:

[0051] Based on local graph data from edge clients, structure-enhanced graph data is constructed by extracting and fusing high-order topology structures;

[0052] Unsupervised training is performed based on the structure enhancement graph data to obtain the gradient update amount of the local model parameters;

[0053] Perform residual sparse compression processing with key layer protection on the gradient update amount to generate sparse gradient information to be uploaded, and accumulate and save the gradient information discarded in this compression process as historical residuals.

[0054] The sparse gradient information is sent to the cloud server so that the cloud server can aggregate information from multiple edge clients and update the global model parameters.

[0055] According to the preset multi-stage scheduling strategy, the training, compression, transmission and aggregation processes are executed iteratively until the model training reaches the preset convergence condition.

[0056] Specifically:

[0057] Step S1: On the edge client, based on the locally held graph data, a high-order correlation matrix is ​​constructed by calculating the triangular motif, and the high-order correlation matrix is ​​weighted and fused with the original graph topology to generate local graph data with enhanced structure.

[0058] Step S2: Based on the local graph data with the enhanced structure, perform unsupervised training using a graph autoencoder model, optimize the model parameters by minimizing the reconstruction loss function, and obtain the updated local model parameters;

[0059] Step S3: Calculate the difference between the local model parameters and the previous round of global model parameters to obtain the initial gradient update amount; combine the local stored historical residual memory, perform residual sparse compression with key layer protection on the initial gradient update amount to generate sparse uploaded gradients, and update the unuploaded gradient information to the historical residual memory.

[0060] Step S4: Send the sparse upload gradient to the cloud server. The server performs weighted aggregation based on the data volume weight of each edge client, updates the global model parameters, and sends them down to each edge client.

[0061] Step S5: According to the preset multi-stage dynamic collaborative scheduling strategy, iteratively execute steps S2 to S4 until the model reaches the convergence condition or the preset training rounds.

[0062] The method for generating the structure-enhanced local graph data in step S1 specifically includes: an adjacency matrix based on the local graph data. Calculate its square matrix The motif matrix is ​​calculated using the Hadamard product. ,in The non-zero elements in the matrix represent the number of triangular structures existing between nodes; for the motif matrix Logarithmic smoothing and normalization are performed to obtain the motif weight matrix; fusion coefficients are then introduced. The motif weight matrix is ​​superimposed onto the original adjacency matrix. The formula above is: This allows for the introduction of higher-order structural semantics while preserving first-order neighbor information.

[0063] The residual sparse compression method with key layer protection in step S3 specifically includes: an error compensation mechanism: adding the current initial gradient update amount to the historical residual memory left over from the previous round to obtain the cumulative gradient update amount, ensuring that untransmitted gradient information is not lost; key layer identification: traversing the parameters of each layer of the model to identify whether it belongs to a "key layer"; the key layer includes the model's output mapping layer (such as the conv2 layer in the code), the bias term, and small layers with fewer than a preset threshold (such as 2000 parameters); and a differentiated compression strategy.

[0064] If it is determined to be a critical layer, compression is not performed. Instead, the complete cumulative gradient update of that layer is directly added to the sparse uplink gradient, and the corresponding residual memory is cleared to zero.

[0065] If it is determined to be a non-critical layer, the number of layers to retain is calculated based on the current compression ratio. Select the one with the largest absolute value. The parameters are added to the sparse uploading gradient, and the remaining unselected parameters are stored in the new historical residual memory for error compensation in the next round.

[0066] Furthermore, the multi-stage dynamic collaborative scheduling strategy described in step S5 specifically includes: a warmup phase: before the start of training. In the first round (e.g., 10 rounds), the compression rate is forced to 0, i.e., full parameter transfer is performed to quickly build the skeleton of the global model; in the second round, the compression rate is forced to 0 and full parameter synchronization is performed every fixed synchronization interval (e.g., 10 rounds) to correct model drift accumulated by sparse communication; in the third round, the compression rate is linearly increased with the number of training rounds until the target value is reached, in the fourth round, in the fifth round, and in the sixth round, the compression rate is reduced at a set step size to achieve fine convergence under communication constraints.

[0067] Example 2

[0068] This embodiment provides a lightweight federated graph autoencoder training method for edge computing, including:

[0069] like Figure 4 As shown, this embodiment is applied to a monitoring terminal (hereinafter referred to as "monitoring terminal") consisting of several industrial CNC machine tools distributed in different workshops. Figure 4 A distributed industrial Internet of Things (IoT) system consisting of edge computing nodes and a cloud-based fault diagnosis center (i.e., the aforementioned cloud aggregation server).

[0070] Objective object: Each monitoring terminal in the system is connected to a CNC machine tool, which includes several key physical components such as spindle, feed axis, tool magazine, and hydraulic pump (corresponding to nodes in the figure).

[0071] Objective Problem: In industrial production, faults between machine tool components are often correlated (for example, spindle vibration can lead to bearing wear, which in turn affects machining accuracy). Accurate diagnosis of such coordinated faults requires the acquisition of high-frequency vibration and current signals. However, the bandwidth of wireless networks (such as 5G / WiFi) in factory settings is limited, making it impossible to upload massive amounts of raw sensor data in real time; simultaneously, fault samples from individual machine tools are scarce, making it difficult to independently train a high-precision diagnostic model.

[0072] The purpose of this solution is to utilize the lightweight federated graph autoencoder method of this invention to collaboratively train a global diagnostic model that can identify "strongly coupled fault modes" between components using the local data of each machine tool without uploading the original sensitive data, and to significantly reduce the network bandwidth consumption caused by model transmission.

[0073] The goal of this embodiment is to train a graph autoencoder model using federated learning to perform feature extraction and unsupervised clustering (such as dividing equipment into communities) on nodes. Specifically, throughout the entire training process, the original sensor data and component topology of each machine tool are strictly stored in the local edge node and are not transmitted over the network. Only the update of model parameters is exchanged, thereby ensuring the privacy and security of factory production data.

[0074] like Figure 1 As shown, the specific implementation process of this embodiment includes the following five steps:

[0075] Step S1: Higher-order enhancement of the mechanical / electrical coupling topology of machine tool components:

[0076] like Figure 1 Step S1 and Figure 2 As shown, real-time operating status data of the machine tool is acquired at each monitoring terminal. Since the physical connection structure of the machine tool (such as bolted connections and gear meshing) is fixed and relatively sparse, in order to capture the implicit vibration transmission or thermal coupling relationship between components, it is necessary to calculate and fuse the triangular phantom (M3).

[0077] (1) Constructing a component association matrix: Suppose a machine tool has Define an adjacency matrix for each key component (such as the spindle, X-axis motor, Y-axis motor, etc.). If components With components If there is a direct mechanical connection or electrical control association, then .like Figure 2 As shown on the left, first calculate square matrix The formula is:

[0078] (1)

[0079] Physical meaning: Indicates components and components The number of indirect coupling paths generated between them through an intermediate component (e.g., motor) coupling The lead screw and the motor are indirectly coupled.

[0080] (2) Extract the closed-loop coupling module (M3): For example Figure 2 As shown in the intermediate computation module, the motif matrix is ​​calculated using the Hadamard Product:

[0081] (2)

[0082] Physical meaning: Non-zero representation component With another component This forms a mechanical closed-loop structure (triangular mold). In rotating machinery, such closed-loop structures (such as the meshing ring of "gear A-gear B-gear C") are usually areas where vibration energy accumulates and faults propagate. The larger the value, the tighter the mechanical coupling in the region, and the higher the probability of a coordinated failure.

[0083] (3) Generate an enhanced fault propagation topology: such as Figure 2 As shown on the right, the phantom matrix Normalized and superimposed on the original connection matrix The above yields an enhanced topology. :

[0084] (3)

[0085] (4)

[0086] in, To incorporate the hyperparameter (0.8 in this example), This refers to the enhanced adjacency structure input into the neural network.

[0087] Technical effect: This allows subsequent neural networks to not only focus on directly connected components, but also to focus on components in strong mechanical coupling closed loops, thus more sensitively capturing resonance or cascading fault characteristics.

[0088] Step S2: Local unsupervised training based on component state features:

[0089] like Figure 1 As shown in step S2, each monitoring terminal utilizes the enhanced topology Train a local GAE fault feature extraction model.

[0090] (1) Encoder forward propagation: A two-layer graph convolutional network (GCN) is used to aggregate component features.

[0091] The first-level formula is:

[0092] (5)

[0093] The second-level formula is:

[0094] (6)

[0095] Where X is the real-time sensor feature matrix of each component of the machine tool (such as vibration frequency and effective current value). These are the weight parameters used in the training process. It is the final output component health status embedding vector containing topological information.

[0096] (2) Decoding and Reconstruction (Decoder): Reconstructing the component coupling structure of the machine tool using a dot product decoder:

[0097] (7)

[0098] in It is the Sigmoid activation function. This represents the predicted probability of association between components.

[0099] (3) Loss Calculation: Calculate the reconstruction loss, with the goal of minimizing the prediction error under normal operating conditions, thereby learning the normal operating mode of the machine tool:

[0100] (8)

[0101] Update local parameters using gradient descent. .

[0102] Step S3: Residual sparse compression adapted to narrow bandwidth environments

[0103] After completing local training, the monitoring terminal needs to upload the model update. To adapt to bandwidth-constrained industrial wireless networks, the following steps are performed: Figure 3 The compression logic shown is as follows:

[0104] (1) Calculate the cumulative gradient:

[0105] like Figure 3 As shown on the far left, the current gradient is calculated. Read the residual memory that was not uploaded in the previous round from local residual memory 201. Calculate the cumulative update amount :

[0106] (9)

[0107] (2) Key layer identification and full retention:

[0108] like Figure 3 As shown in the diamond shape, the model layers are traversed. For key layers (including the output layer that determines the dimensions of fault features, the bias term Bias that affects the activation threshold, and the small parameter layer):

[0109] If it is determined to be a critical layer, then no compression will be performed, and the upload ladder will be directly set. And clear the residual of that layer to zero. This is to prevent the fault judgment benchmark from drifting due to compression.

[0110] (3) Top-k sparsity of non-critical layers:

[0111] If a layer is determined to be non-critical (such as input layer weights), set the compression ratio. (e.g., 0.9). Calculate the threshold. Only retain the first one with the largest change. part:

[0112] (10)

[0113] (11)

[0114] (12)

[0115] In other words, only significant parameter changes reflecting emerging fault trends are uploaded; minor changes are truncated and stored. In the next round of communication, the process continues to accumulate, achieving unbiased remote diagnostic model updates.

[0116] Step S4: Cloud-based fault mode aggregation:

[0117] like Figure 1 Step S4 and Figure 4 As shown, the cloud-based fault diagnosis center 100 receives sparse gradients uploaded by each monitoring terminal. The learning outcomes from various machine tools were fused using the FedAvg algorithm.

[0118] (13)

[0119] The number of connected machine tool components allows the cloud to generate a general fault diagnosis model that integrates various working conditions and fault characteristics of multiple machine tools.

[0120] Step S5: Dynamic scheduling adapted to the production cycle:

[0121] like Figure 1 As shown in step S5, the training strategy is adjusted according to the factory's production cycle:

[0122] 1) Warm-up Phase: Corresponds to the trial operation period of a new production line. Before... Wheel Forced (Full data transmission) to quickly establish a baseline health model.

[0123] 2) Periodic Full Synchronization: Corresponds to the regular equipment calibration period. Every [period]... Wheel forced again This eliminates model bias caused by long-term sparse communication.

[0124] 3) Adaptive decay: For stable production periods. In conjunction with the StepLR scheduler, the learning rate is multiplied by a decay factor every fixed number of rounds. Maintaining "fine-tuning" of the model with extremely low bandwidth usage, while continuously monitoring the chronic degradation of the equipment.

[0125] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0126] On the other hand, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0127] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lightweight federated graph autoencoder training method for edge computing, characterized in that, include: Based on local graph data from edge clients, structure-enhanced graph data is constructed by extracting and fusing high-order topology structures; Unsupervised training is performed based on the structure enhancement graph data to obtain the gradient update amount of the local model parameters; Perform residual sparse compression processing with key layer protection on the gradient update amount to generate sparse gradient information to be uploaded, and accumulate and save the gradient information discarded in this compression process as historical residuals. The sparse gradient information is sent to the cloud server so that the cloud server can aggregate information from multiple edge clients and update the global model parameters. According to the preset multi-stage scheduling strategy, the training, compression, transmission and aggregation processes are executed iteratively until the model training reaches the preset convergence condition.

2. The method according to claim 1, characterized in that, The method of constructing structure-enhanced graph data by extracting and fusing higher-order topological structures includes: Based on the adjacency matrix of the local graph data, a square matrix representing the second-order path is obtained by multiplying the adjacency matrix by itself. The square matrix is ​​multiplied element-wise with the original adjacency matrix to obtain the motif matrix representing the triangular relationship structure between nodes. The motif matrix is ​​smoothed and normalized to obtain the motif weight matrix; The motif weight matrix and the original adjacency matrix are weighted and superimposed according to an adjustable fusion coefficient to generate the final structure enhancement graph data.

3. The method according to claim 2, characterized in that, The smoothing and normalization of the motif matrix specifically includes: First, a smoothing operation is performed on each element value in the motif matrix, which is then incremented by one and the logarithm is taken. Then, the values ​​of each element in the smoothed matrix are divided by its global maximum value to complete the normalization.

4. The method according to claim 1, characterized in that, Perform residual sparse compression with critical layer protection on the gradient update amount, including: Read the historical residual memory left over from the previous training round stored locally; The initial gradient update amount calculated in the current round is added to the historical residual memory to obtain the cumulative gradient update amount; Identify the parameters belonging to a preset key layer in the accumulated gradient update amount, wherein the key layer includes at least the model's output mapping layer and all bias term parameters; The identified key layer parameters are fully retained, their complete values ​​are directly added to the sparse gradient information to be uploaded, and their corresponding historical residual memory is cleared. For the cumulative gradient update amount of non-critical layers, according to the preset compression ratio, only the first part of the parameter values ​​with the largest absolute value is retained and sparse gradient information is added. The parameter values ​​that were not selected and retained in the non-critical layers are saved as new historical residual memories for compensation in the next training round.

5. The method according to claim 1, characterized in that, The multi-stage scheduling strategy includes: During the initial warm-up phase, gradient compression is disabled, and a full transfer of all parameters is performed. During the regular training phase following the warm-up phase, full parameter synchronization rounds are periodically inserted to correct accumulated errors. In regular rounds of non-full synchronization, gradient compression is enabled, and the compression ratio is gradually increased to the target value as the training progresses. Throughout the training process, the model's learning rate is gradually reduced according to a predetermined schedule.

6. The method according to claim 1, characterized in that, The compression ratio gradually increases as the training progresses, and the increase is based on the ratio of the current training round to the total training rounds, increasing linearly from the initial value to the target compression ratio.

7. The method according to claim 1, characterized in that, The unsupervised training based on structure-enhanced graph data is achieved through a graph autoencoder model, including: The input augmented graph structure and node features are encoded using a graph convolutional network to obtain low-dimensional embedding vectors for the nodes. The original graph connectivity is reconstructed by performing inner product operations between the embedded vectors and transforming them using an activation function. The parameters of the graph autoencoder model are updated by minimizing the difference loss between the reconstructed graph and the original graph.

8. The method according to claim 1, characterized in that, The cloud server performs aggregation in the following way: Assign appropriate weights to each edge client based on the size of its local graph data. The weights are used to perform a weighted average of the sparse gradient information received from each client, and the result of the weighted average is used to update the global model.

9. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, When the processor executes the computing program, it implements the method of any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-8.

Images