An industrial equipment operation state monitoring method based on federal forward pulse learning

Abstract

The application discloses an industrial equipment operation state monitoring method based on federal forward pulse learning, and belongs to the technical field of industrial edge intelligence and federal learning. The method comprises the following steps: a federal training environment composed of a center server and a plurality of industrial terminals is constructed; each industrial terminal constructs a positive supervision input and a negative supervision input under a label condition; a rate coding is used to convert a continuous value input into a pulse sequence, time-aggregated pulse activations of each layer and hierarchical scores are calculated, a terminal-layer corresponding aggregation weight is calculated by the center server according to a sample weight and a loss of each layer, and a weighted update is performed; a candidate operation state category is enumerated, and the sum of hierarchical scores of each layer is compared, and an equipment operation state monitoring result is output. Through the hierarchical score mechanism and the forward-forward local update strategy, the application reduces the activation storage pressure in the training stage of the industrial terminal, and is suitable for the operation state monitoring scene of an industrial production line, an equipment cluster, a control system or an edge node of an industrial internet of things.

Description

Technical Field

[0001] This invention relates to the fields of federated learning and artificial intelligence, specifically to a federated forward impulse learning method for industrial edge scenarios. Background Technology

[0002] The application of industrial systems in fields such as intelligent manufacturing and the Industrial Internet of Things (IIoT) is becoming increasingly widespread. Monitoring the operational status of industrial equipment is crucial for ensuring production safety and improving efficiency. With the development of edge computing technology, deploying intelligent models to industrial edge terminals has become a trend. Federated learning, as a privacy-preserving distributed machine learning paradigm, collaboratively optimizes a global model by training it locally and interacting only with its parameters, thus ensuring the utilization of multi-party industrial data while protecting privacy. However, when dealing with complex industrial edge scenarios, it often faces problems such as high training memory pressure, difficulty in model convergence, and poor generalization ability. In real-world industrial environments, building low-memory, high-precision federated learning models offers significant benefits. Therefore, the research results of this invention have broad application prospects.

[0003] However, existing technologies have significant limitations when dealing with industrial edge scenarios. First, most existing federated learning methods use backpropagation for local training, requiring the storage of activation values ​​for each layer for gradient calculation, resulting in high memory pressure on industrial terminals. Second, spiking neural networks, as a neural network model with event-driven and low-power characteristics, rely on backpropagation for local training, limiting their deployment on resource-constrained industrial edge devices. Furthermore, due to the non-independent and identically distributed nature of industrial data, traditional federated averaging methods only aggregate the entire model based on sample size, leading to difficulties in global model convergence. Therefore, proposing a federated learning architecture that can reduce training memory pressure, avoid backpropagation dependency, and improve aggregation performance has become an important topic in the field of federated learning.

[0004] This invention designs a federated forward impulse learning method for industrial edge scenarios, achieving low-memory, high-precision monitoring of industrial equipment operating status. This method enhances the training signal by designing a labeled conditional positive and negative supervised input construction mechanism, converts continuous value inputs into impulse sequences using rate coding, employs forward-forward loss to perform local updates within layers to avoid backpropagation dependencies, and improves the collaborative modeling effect under heterogeneous terminals through a hierarchical scoring mechanism and a hierarchical weighted aggregation strategy. In the federated collaborative training process, this method overcomes the limitations of existing methods in industrial edge scenarios, achieving a low-memory, low-power federated model training method, and providing an effective approach to improve the intelligent monitoring capabilities of industries in complex environments. Summary of the Invention

[0005] To address the problems of existing federated learning methods in industrial edge scenarios, such as high training memory pressure, spiking neural networks relying on backpropagation, and inconsistent aggregation caused by non-independent and identically distributed data, resulting in low global model accuracy and deployment difficulties.

[0006] The technical solution adopted in this invention is a method for monitoring the operating status of industrial equipment based on federated forward impulse learning, comprising the following steps:

[0007] (1) Construct a terminal collaborative training environment in an industrial edge scenario: Initialize global spiking neural network parameters, set training rounds, local forward update steps, local learning rate, server-side step size, rate coding time window and scoring threshold; define local industrial datasets and industrial operation status category sets for each industrial terminal; at the beginning of each round of communication, send the current global model parameters to all industrial terminals.

[0008] (2) Perform pulse coding: construct label conditional positive supervision input and label conditional negative supervision input for local samples to form positive supervision sample set and negative supervision sample set respectively; perform rate coding on positive supervision input and negative supervision input to convert continuous value input into pulse sequence with pulse firing frequency proportional to input value, and input the encoded pulse sequence into pulse neural network based on pulse timing coding;

[0009] (3) Perform local forward updates within the layer: perform temporal aggregation on the output pulses of each layer within the rate coding time window to obtain temporally aggregated pulse activations; calculate the layer-level score based on the temporally aggregated pulse activations; define the forward-forward supervision objective, construct the forward-forward loss function for each layer, and define the local objective function for each layer accordingly; before the start of each round of local training, receive the global layer parameters and calculate the local objective function value of the current layer; within each layer, with the local objective function of the layer as the optimization objective, perform several forward-forward local updates to update the parameters of each layer;

[0010] (4) Update global model parameters: The central server receives the hierarchical model update direction and hierarchical loss uploaded by each industrial terminal, calculates the aggregation weight of each terminal in each layer, performs weighted update of the global model parameters of each layer with the aggregation weight, and sends the updated global model parameters to all industrial terminals as the initial model for the next round of federated training. Steps (2) to (3) are repeated until the preset communication round is reached.

[0011] (5) Perform industrial equipment operation status monitoring: After training, save the global model parameters, deploy the obtained spiking neural network model to the industrial terminal; enumerate candidate operation status categories, construct the label condition candidate inputs corresponding to each candidate category, calculate the sum of scores at each level, and output the industrial operation status monitoring results according to the candidate operation status category with the largest sum of level scores.

[0012] Further, the construction of the label conditional positive supervision input and label conditional negative supervision input in step (2) is specifically as follows: the one-hot vector corresponding to the true category of the local sample is used as the positive supervision input, and the one-hot vector corresponding to the error category obtained by sampling from the global category set with uniform probability is used as the negative supervision input.

[0013] Furthermore, the rate encoding described in step (2) converts the continuous value input into a pulse sequence in which the pulse firing frequency is proportional to the input value, and adopts a pulse potential update model for spiking neurons based on pulse timing encoding, wherein the relationship between synaptic input, membrane potential before firing pulse, binary output pulse, membrane potential after firing or resetting, firing threshold and resetting potential is described by the membrane potential update equation, pulse firing determination equation and resetting equation.

[0014] Furthermore, the time aggregation of the output pulses of each layer described in step (3) is to average the binary output pulses of each layer at all time steps within the rate coding time window to obtain the time aggregation pulse activation vector.

[0015] Furthermore, the hierarchical scoring in step (3) is obtained by dividing the square of the second norm of the time-aggregated pulse activation vector of each layer by the width of the neuron in that layer.

[0016] Further, the forward-forward loss in step (3) is constructed for positive and negative samples respectively using cross-entropy loss functions based on the difference between hierarchical scores and score thresholds; the hierarchical local objective function is the expectation of the forward-forward loss on the positive and negative supervised sample sets.

[0017] Furthermore, the intra-layer forward-forward local update described in step (3) takes the hierarchical local objective function as the optimization objective and uses the gradient descent method to independently update the parameters within the layer, without relying on backpropagation to transfer gradients across layers.

[0018] Further, the step (4) of calculating the aggregate weight of each terminal in each layer is specifically as follows: the normalized aggregate weight is calculated by taking the product of the sample size weight of each terminal and the reciprocal of the local objective function value of the terminal in the current layer as the numerator and the sum of the above products of all terminals as the denominator.

[0019] Furthermore, the weighted update of the global model parameters of each layer in step (4) is to perform a weighted summation of the update directions of the hierarchical models uploaded by each terminal using the aggregated weights, multiply by the server-side step size, and then subtract it from the current global layer parameters to obtain the updated global layer parameters.

[0020] Further, the output of the industrial operation status monitoring results in step (5) specifically involves: inputting the candidate label conditions corresponding to each candidate operation status category into the trained spiking neural network, calculating the sum of the layer scores of all layers, and selecting the candidate category with the largest sum of scores as the final output operation status category; when the output category is normal operation status, it is determined that the industrial equipment is in normal operation status; when the output category is abnormal operation status, a corresponding abnormal prompt is generated.

[0021] This invention proposes a federated forward spiking learning method for monitoring the operational status of industrial equipment, significantly improving the efficiency and accuracy of collaborative training of federated models in industrial edge scenarios. This method integrates a forward-forward local update mechanism with a hierarchical scoring aggregation strategy, effectively solving the core challenges of traditional federated learning methods in industrial edge scenarios, such as high training memory pressure, backpropagation dependence, and data heterogeneity leading to model performance degradation: high activation storage pressure, difficulty in deploying spiking neural networks, and poor aggregation effects. This method reduces the activation storage pressure during the training phase of industrial terminals, improves the collaborative modeling effect under heterogeneous industrial terminals, and can well complete the collaborative training task of distributed industrial terminals. This invention is applicable to operational status monitoring scenarios of industrial production lines, equipment clusters, control systems, or industrial IoT edge nodes. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of an industrial edge federated training environment.

[0023] Figure 2 This is an overall flowchart of an industrial edge modeling method based on federated unidirectional neuromorphic learning.

[0024] Figure 3 This is a schematic diagram of the positive and negative input structure of the tag conditions on the industrial terminal side and the forward local update of the pulse.

[0025] Figure 4 This is a diagram illustrating the weighted aggregation of losses at the central server level. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] A method for monitoring the operating status of industrial equipment based on federated forward impulse learning is characterized by constructing a terminal collaborative training environment in an industrial edge scenario, performing impulse coding, performing local forward updates within the layer, updating global model parameters, and performing industrial equipment operating status monitoring.

[0028] (1) Constructing a terminal collaborative training environment in industrial edge scenarios

[0029] Define the global spiking neural network parameters and initialize them as follows: And set training rounds Local forward update of steps Local learning rate Server-side step size Rate coding time window Scoring threshold ;Set the first The local industrial dataset for each industrial terminal is Total number of samples Sample size weights Define the set of industrial operating status categories as ,in This indicates the number of industrial operation status categories, which include six categories: normal operation, abnormal operation, fault warning, fault diagnosis, quality risk, and industrial network abnormality. At the start of the t-th round of communication, the current global model parameters are sent to all 50 industrial terminals.

[0030] (2) Execute pulse coding

[0031] Given local samples Constructing label conditions for positive supervision of input For category vector Construct labeled conditional negative supervised input The error category is obtained by sampling from the global category set Y with probability 1 / (|Y|−1). vector :

[0032] Obtain the positive supervised sample set of the kth industrial terminal. and negative supervision sample set ;

[0033] At the kth industrial terminal, for and Rate encoding is performed to transform the continuous input into a pulse sequence whose pulse firing frequency is proportional to the input value. The encoded pulse sequence is then fed into a pulse neural network based on pulse timing encoding. Synaptic input, The membrane potential before the pulse is emitted. It is a binary output pulse. The membrane potential after discharge or reset. The threshold for issuance, This is the reset potential.

[0034]

[0035] Where Θ is the step function and f is the membrane potential update function;

[0036] (3) Local forward update within the execution layer

[0037] For the k-th industrial terminal Layer Each time step outputs a pulse to perform time aggregation, resulting in a time-aggregated pulse activation. The calculation formula is:

[0038] based on Calculate the level score:

[0039]

[0040] in, Indicates the first Layer width, express The There are 1 component; define the forward-forward supervision target s, where positive samples correspond to s=1 and negative samples correspond to s=0, and the 1st component... The layer forward-forward loss is:

[0041] Define the local objective function corresponding to the l-th layer of the k-th industrial terminal. for:

[0042] in, The function represents a curve, and its formula is:

[0043]

[0044] Map the hierarchical scores to the (0,1) probability interval; receive global layer parameters before each round of local training. And calculate ;

[0045] In the Within the layer, using the local objective function of the layer in formula (8) as the optimization objective, three forward-forward local updates are performed, with the parameter update formula as follows:

[0046] (4) Update global model parameters

[0047] After receiving the hierarchical model orientation and hierarchical loss uploaded by each industrial terminal, the central server calculates the corresponding aggregate weights for each terminal and layer. :

[0048] Central server As the weight, for the th The layer-wide model parameters are updated using a weighted method, where... Indicates the model update direction of the k-th industrial terminal at layer l:

[0049] Distribute updated global model parameters All 50 industrial terminals are used as the initial model for the next round of federated training, and steps (2) to (3) are repeated until the preset number of communication rounds is reached. ;

[0050] (5) Perform industrial equipment operation status monitoring

[0051] Save global model parameters after training. The resulting spiking neural network model is then deployed to industrial terminals; candidate operating state categories are enumerated. Construct labeled conditional candidate inputs Calculate the sum of scores for each level and output the industrial operation status monitoring results according to the candidate operation status category with the largest sum of level scores.

[0052]

[0053] When the output category is normal operation, the industrial control system is determined to be in normal operation; when the output category is abnormal, an abnormal prompt is generated, thereby completing the equipment operation status monitoring for industrial edge scenarios.

[0054] The system constructs labeled positive and negative supervised inputs; calculates temporally aggregated pulse activation, layer score, and layer forward-forward loss within each layer of the spiking neural network, and performs local forward updates within each layer; calculates the sample size weights and layer-by-layer aggregated weights of the layer loss, and performs weighted updates of the global model parameters for each layer; enumerates candidate operating state categories and compares the sum of layer scores, outputs the operating state category, and generates an anomaly prompt when an abnormal state is identified, thereby realizing the monitoring of industrial equipment operating status for industrial edge scenarios.

Claims

1. A method for monitoring the operating status of industrial equipment based on federated forward impulse learning, characterized in that, Construct a terminal collaborative training environment in an industrial edge scenario, perform pulse coding, perform local forward updates within the layer, update global model parameters, and perform industrial equipment operation status monitoring; (1) Construct a terminal collaborative training environment in industrial edge scenarios; Define the global spiking neural network parameters and initialize them as follows: And set training rounds Local forward update of steps Local learning rate Server-side step size Rate coding time window Scoring threshold ;Set the first The local industrial dataset for each industrial terminal is Total number of samples Sample size weights Define the set of industrial operation status categories as ,in This indicates the number of industrial operation status categories, which include six categories: normal operation, abnormal operation, fault warning, fault diagnosis, quality risk, and industrial network abnormality. At the start of the t-th round of communication, the current global model parameters are sent to all 50 industrial terminals. (2) Execute pulse coding; Given local samples Constructing label conditions for positive supervision of input For category vector Construct labeled conditional negative supervised input The error category is obtained by sampling from the global category set Y with probability 1 / (|Y|−1). vector : Obtain the positive supervised sample set of the k-th industrial terminal. and negative supervision sample set ; At the kth industrial terminal, for and Rate encoding is performed to transform the continuous input into a pulse sequence whose pulse firing frequency is proportional to the input value. The encoded pulse sequence is then fed into a pulse neural network based on pulse timing encoding. Synaptic input, The membrane potential before the pulse is emitted. It is a binary output pulse. The membrane potential after discharge or reset. The threshold for issuance, This is the reset potential. ； Where Θ is the step function and f is the membrane potential update function; (3) Local forward update within the execution layer; For the k-th industrial terminal Layer Each time step outputs a pulse to perform time aggregation, resulting in a time-aggregated pulse activation. The calculation formula is: based on Calculate the level score: ； in, Indicates the first Layer width, express The There are 1 component; define the forward-forward supervision target s, where positive samples correspond to s=1 and negative samples correspond to s=0, and the 1st component... The layer forward-forward loss is: Define the local objective function corresponding to the l-th layer of the k-th industrial terminal. for: in, The function represents a curve, and its formula is: ； Map the hierarchical scores to the (0,1) probability interval; receive global layer parameters before each round of local training. And calculate ; In the Within the layer, using the local objective function of the layer in formula (8) as the optimization objective, three forward-forward local updates are performed, with the parameter update formula as follows: (4) Update global model parameters; After receiving the hierarchical model orientation and hierarchical loss uploaded by each industrial terminal, the central server calculates the corresponding aggregate weights for each terminal and layer. : Central server As the weight, for the th The layer-wide model parameters are updated using a weighted method, where... Indicates the model update direction of the k-th industrial terminal at layer l: Distribute updated global model parameters All 50 industrial terminals are used as the initial model for the next round of federated training, and steps (2) to (3) are repeated until the preset number of communication rounds is reached. ; (5) Perform industrial equipment operation status monitoring; Save global model parameters after training. The resulting spiking neural network model is then deployed to industrial terminals; candidate operating state categories are enumerated. Construct labeled conditional candidate inputs Calculate the sum of scores for each level and output the industrial operation status monitoring results according to the candidate operation status category with the largest sum of level scores. ； When the output category is normal operation, the industrial control system is determined to be in normal operation; when the output category is abnormal, an abnormality prompt is generated, thereby completing the equipment operation status monitoring for industrial edge scenarios.

2. The industrial equipment operation status monitoring method based on federated forward impulse learning according to claim 1, characterized in that: The system constructs labeled positive and negative supervised inputs; calculates temporally aggregated pulse activation, layer score, and layer forward-forward loss within each layer of the spiking neural network, and performs local forward updates within each layer; calculates the sample size weights and layer-by-layer aggregated weights of the layer loss, and performs weighted updates of the global model parameters for each layer; enumerates candidate operating state categories and compares the sum of layer scores, outputs the operating state category, and generates an anomaly prompt when the state is abnormal, thereby realizing the monitoring of industrial equipment operating status for industrial edge scenarios.

Images