An industrial robot stream anomaly detection method and device based on dynamic graph enhanced normalized stream

Abstract

This invention discloses a method and device for industrial robot streaming anomaly detection based on dynamic graph-enhanced normalized flow, belonging to the interdisciplinary field of industrial control and artificial intelligence. Addressing the problems of high latency and difficulty in capturing cross-sensor dynamic dependencies caused by existing flow-based models relying on offline full-trajectory evaluation, this invention employs a fixed-length sliding window to perform online partitioning of high-frequency multi-channel sensor data. A dynamic graph enhancement module is introduced into the coupled sub-network of the normalized flow to adaptively calculate channel correlations within the current window, constructing a soft adjacency matrix and jointly extracting local temporal patterns and dynamic cross-sensor dependencies. Finally, a Causal-Max strategy is used, combining causal smoothing and maximum value extraction, to convert window-level scores into stable trajectory-level anomaly scores. This invention achieves low-latency, low-resource-consumption online anomaly early warning, significantly improving the accuracy of multivariate temporal anomaly detection in complex industrial environments.

Description

Technical Field

[0001] This invention relates to the intersection of industrial control and artificial intelligence, specifically to a high-frequency streaming anomaly detection technology for multivariate time series. Background Technology

[0002] Industrial robots are core components of modern automated production lines. Their high-intensity and continuous operation makes them susceptible to physical wear and external interference. In complex collaborative environments, undetected mechanical anomalies not only reduce production yield but also seriously threaten human and machine safety. Therefore, unsupervised anomaly detection based on high-frequency internal operating data (multi-channel sensor time series) has become an important research topic in the field of industrial equipment health management and intelligent operation and maintenance.

[0003] In recent years, deep generative models, represented by normalizing flows, have shown great potential in multivariate time series anomaly detection, providing accurate log-likelihood estimates and enabling anomaly detection without explicitly pre-setting data distributions. However, when faced with the high-frequency, continuous data streams of real-world industrial robots, existing normalizing flow-based anomaly detection methods still suffer from the following significant limitations: First, existing detection frameworks (such as MVT-Flow) are typically designed for offline trajectory-level evaluation. This post-processing paradigm requires caching the complete action cycle (complete trajectory) sequence before computation and evaluation can be performed. In low-latency industrial streaming monitoring scenarios, this offline mechanism introduces unacceptable lag delays, failing to meet the requirements for real-time early warning.

[0004] Secondly, existing multivariate time series flow models typically rely solely on standard one-dimensional convolutions (1D CNNs) for local temporal pattern extraction within coupled subnetworks, supplemented by random permutation operations. While this structure is easy to implement, its ability to extract cross-channel (cross-sensor) structured dependencies is very limited, failing to capture the dynamically changing coupling dependencies between sensors under different operating states and control commands of industrial robots.

[0005] Finally, in a streaming online detection environment, if a sliding window mechanism is used, the same trajectory will inevitably generate multiple consecutive window-level anomaly scores. How to suppress the random fluctuations of high-frequency scores and stably aggregate these local scores into reliable trajectory-level anomaly decisions without relying on future data (satisfying causality) remains a problem that has not yet been effectively solved in existing research. Summary of the Invention

[0006] This invention aims to overcome the shortcomings of existing technologies and provide a method and device for industrial robot streaming anomaly detection based on dynamic graph-enhanced normalized flow. The invention primarily addresses the technical problems of existing detection methods, such as high warning delays due to reliance on offline full-trajectory evaluation, difficulty in adaptively capturing dynamic time-varying dependencies across sensors, and the difficulty in stably mapping streaming window scores to global anomaly decisions.

[0007] To address the aforementioned technical problems, this invention provides a method for detecting flow cytometry anomalies in industrial robots based on dynamic graph-enhanced normalized flow, comprising the following steps: S1. Continuously acquire the multi-channel sensor time series during the operation of the industrial robot, and use a sliding window of preset length to divide the multi-channel sensor time series online to obtain the local window feature tensor of the current time step as streaming input data; S2. Input the streaming input data into a pre-trained dynamic graph augmented normalized flow model; wherein, the coupled sub-network of the dynamic graph augmented normalized flow model includes a dynamic graph augmentation module, which adaptively constructs a soft adjacency matrix representing the cross-sensor coupling strength based on the channel correlation within the local window feature tensor of the current time step, and jointly extracts local temporal patterns and dynamic cross-sensor dependency features during the reversible transformation process, and outputs the window-level anomaly score corresponding to the current time step; S3. Cache the window-level anomaly scores output at the current time step and historical time steps, and perform causal smoothing on the window-level anomaly score sequence sorted by time using a fixed-length causal smoothing window to suppress high-frequency fluctuations. S4. Extract the maximum value from the window-level anomaly score sequence after causal smoothing, and use it as the trajectory-level anomaly score of the industrial robot under the current trajectory. S5. Compare the trajectory-level anomaly score with a preset alarm threshold. If the trajectory-level anomaly score is greater than the alarm threshold, output an abnormal state warning signal.

[0008] Compared with existing technologies, the present invention has the following significant advantages: (1) Breaking the latency bottleneck of offline evaluation and realizing low-overhead streaming online detection: This invention reconstructs the full trajectory detection paradigm into a streaming processing architecture based on a fixed-length sliding window. The model does not need to wait for the entire working cycle to end during inference, so that memory usage and single-step computation do not increase with the total trajectory length. The latency of a single forward pass reaches the millisecond level, which meets the stringent requirements of resource-constrained equipment and low-latency alarms in industrial production.

[0009] (2) No need for manual topological priors, significantly improving the modeling ability of dynamic dependencies and the anomaly recognition rate: This invention innovatively embeds a state-dependent implicit dynamic graph module into the coupling transformation layer of the normalized flow. This module overcomes the limitations of traditional one-dimensional convolution and can adaptively calculate the soft adjacency matrix based solely on the features of the current window. This dynamic graph enhancement mechanism can accurately capture the cross-sensor coupling changes of industrial robots under different operating states, significantly improving the model's joint distribution fitting ability, and enabling the system to have higher sensitivity and accuracy when dealing with subtle anomalies.

[0010] (3) Balancing causality and decision stability in online detection: This invention designs a Causal-Max trajectory-level score aggregation strategy. By performing fixed-window causal smoothing on time-sorted window-level scores, it effectively filters out random fluctuations caused by local environmental noise; at the same time, it retains the main abnormal features by combining a maximum value extraction strategy. This mechanism uses only historical and current data throughout the process, strictly meeting the temporal causality constraints in streaming deployment, and realizing a reliable mapping from local abnormal evidence to global stable alarms. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 The overall flowchart of the industrial robot flow anomaly detection method based on dynamic graph-enhanced normalized flow provided in the embodiments of the present invention is shown below. Figure 2 This is a schematic diagram of the internal structure of the dynamic graph enhancement module provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention; Figure 4 This is a bar chart comparing the anomaly detection performance of different algorithm models in the embodiments of the present invention; Figure 5 This is a line graph showing the trend of detection accuracy as a function of the sliding window length in an embodiment of the present invention. Figure 6 This is a combined trend diagram of system resource overhead (video memory and latency) as a function of the sliding window length in an embodiment of the present invention.

[0013] Figure 1This is an overall flowchart of a method for detecting flowing anomalies in industrial robots based on dynamic graph-enhanced normalized flow, provided as an embodiment of the present invention. The detection phase of this method (i.e., the forward inference process of the model) specifically includes the following steps: S1: Continuously acquire multi-channel sensor time series and perform online streaming partitioning using a sliding window of preset length to extract local window feature tensors.

[0014] In continuous operation scenarios of industrial robots, the underlying sensor network (including but not limited to servo motor current sensors, joint torque sensors, position encoders, etc.) operates at a fixed sampling frequency. Continuously collect operational status data. Let the original multivariable sensor continuous observation sequence be... ,in In absolute physical time Collected Feature vectors of each sensor channel This represents the total duration of a single operation trajectory. To achieve streaming input, the system maintains a first-in, first-out (FIFO) data buffer in memory. The frame length of the observation window is defined as... The step span is During the first day of equipment operation In each real-time inference step, the system extracts the time index from the buffer. The latest sensor data within the closed interval are stacked and merged to generate a size of Local window feature tensor , which serves as the sole input to this time step network.

[0015] Overcoming causal delay: Traditional offline models must wait for the entire trajectory The tensor can only be generated for unified evaluation after data acquisition is complete, which means a lag of tens of seconds in actual production lines. This step uses local sliding truncation to ensure strict time synchronization between the anomaly detection system and the robot's physical movements.

[0016] Space complexity reduction: This reduces the size of the feature maps that need to reside in GPU memory during model inference from a size that grows infinitely with the trajectory length. Forced truncation and fixation to constant level This reconstruction of the underlying physical layer ensures that the algorithm can run stably and without overflowing, even on industrial edge devices with very limited video memory, such as the NVIDIA Jetson Nano.

[0017] S2: Input the local window feature tensor into the dynamic graph augmented normalized flow model, construct the soft adjacency matrix and jointly extract features, and output the window-level negative log-likelihood (NLL) anomaly score.

[0018] Given the above local window feature tensor The core task of normalizing flows models is to learn an invertible bijective function that transforms a complex observation space into a simple prior distribution space (such as the standard normal distribution). In the model's coupling block, the input hidden feature tensor ( (To hide the number of channels) is divided in half according to the channel dimension. and To achieve stronger information fusion in affine coupling transformations, this invention introduces a dynamic graph enhancement module. This module comprises two parallel feature extraction branches: (1) Temporal extraction branch: Through several one-dimensional convolutional layers (1D Temporal CNN), along the time dimension Perform sliding convolution to extract the local trends and evolution patterns of each sensor on the time axis.

[0019] (2) Dynamic graph propagation branch: This branch does not rely on any prior physical topology (because the force transmission relationship between joints changes dynamically under different robot postures), but is driven by data adaptation. The specific calculation method is as follows: First, compute the current feature tensor. Rather than transpose The inner product is used to evaluate the instantaneous correlation among all channels; Secondly, to prevent the inner product value from becoming too large at higher dimensions, it is divided by the scale factor. Scaling (where) (A learnable temperature parameter used to control the smoothness of the distribution). Finally, the Softmax activation function is applied to the feature dimension to generate a size of soft adjacency matrix Its mathematical expression is:

[0020] Subsequently, this soft adjacency matrix was used Perform graph convolution propagation on the features.

[0021] Fusion and Output: The outputs of the temporal extraction branch and the dynamic graph propagation branch are fused using residual addition, serving as inputs to the scaling and translation factors of the affine coupling layer. After multiple reversible transformations, the features are mapped to the latent space, and the negative log-likelihood (NLL) of the current features under the prior distribution is calculated. This NLL value is the window-level anomaly score for the current time step.

[0022] Adaptive capture of implicit coupling: Industrial robots are complex electromechanical coupling systems. An anomaly in one joint can be implicitly transmitted to the current sensors of other motors through the mechanical structure. Traditional 1D CNNs can only detect the temporal changes of a single sensor, while the soft adjacency matrix of this invention... It can detect such subtle changes in cross-sensor coupling in real time in the feature space.

[0023] Emphasizing Dynamics: It is worth emphasizing that, unlike traditional static graphs built manually based on offline expert experience, and unlike time-varying graphs that evolve according to preset rules, the soft adjacency matrix constructed in this invention is entirely driven by real-time data and evolves dynamically window-by-window. This means that the transient topology of the robot at every tiny time step and under every different working condition can be independently captured, thereby achieving highly adaptive modeling of implicit dependencies across sensors.

[0024] No prior manual intervention required: Traditional graph neural networks (such as ST-GCN) require manually predefined adjacency matrices, which are extremely difficult to define precisely in complex working conditions. The dynamic computation method proposed in this step is entirely driven by real-time data, enhancing the model's generalization ability and robustness across different robotic arm models and processing tasks.

[0025] S3: Cache the window-level anomaly scores output at the current time step and historical time steps, and perform causal smoothing on the window-level anomaly score sequence sorted by time using a fixed-length causal smoothing window.

[0026] Because the aforementioned steps employ a high-frequency sliding window mechanism, for the same continuous trajectory of the robot, the model will continuously output a series of window-level negative log-likelihood (NLL) scores over time, denoted as the time-ordered sequence. .

[0027] In actual industrial workshops, sensor stream data is highly susceptible to local transient noise such as electromagnetic interference and instantaneous mechanical vibrations, which can cause severe and occasional jitter in the high-frequency output window-level fractional sequence.

[0028] To reduce noise in this high-frequency sequence while ensuring strict causality in online detection, this embodiment introduces a physical-time decoupled causal smoothing algorithm.

[0029] First, smooth the window steps. With absolute physical time receptive field Phase decoupling. Let the sensor sampling frequency be... The sliding step size is The number of steps for smoothing causality Dynamic calculation is as follows:

[0030] Using this formula, the algorithm can adaptively maintain a consistent physical smoothing duration (e.g., optimal setting) even under different devices, different sampling rates, or different overlap rates. (1.5 seconds).

[0031] Subsequently, local causal smoothing is applied to the time-sorted window anomaly score sequence, and the smoothing score is calculated. The mathematical logic is as follows:

[0032] The smoothing process described above involves maintaining a length of [value] in memory. This is achieved using a sliding queue.

[0033] Strict adherence to temporal causality: Traditional signal smoothing algorithms require "future" data points to calculate the current point, which is absolutely infeasible in real-time streaming detection. The formula in this invention strictly limits the upper limit of the summation interval to the current time. The lower limit is a historical moment. The entire process ensures that "future information leakage" will never occur, thus meeting the logical constraints of industrial online deployment.

[0034] Physical-temporal receptive field decoupling: by introducing and The adaptive calculation formula decouples the model hyperparameters from the underlying hardware sampling rate, improving the method's portability and engineering robustness across platforms and sensor models.

[0035] S4: Extract the maximum value from the window-level anomaly score sequence after causal smoothing, and use it as the trajectory-level anomaly score of the industrial robot under the current trajectory.

[0036] In a streaming online detection architecture, because the system cannot predict when the current operation trajectory will end, it cannot perform global sorting and extraction after all scores are generated, as in traditional offline models. To accommodate the continuous input of streaming data, this embodiment allocates a fixed-size register variable in memory to maintain the dynamic highest anomaly intensity (i.e., trajectory-level anomaly score) under the current running trajectory. ).

[0037] The specific operation is as follows: at the beginning of each new trajectory, Initialize to a minimum value (or zero). With each time step... As the process progresses, the anomaly decision-making module continuously receives the latest smoothing score from step S3. The system employs a rolling update mechanism, executing the following state transition equation in real time at each time step:

[0038] in, Indicates the first The global maximum anomaly score updated at each time step. This represents the largest outlier score in history up to the previous time step.

[0039] Through this state transition mechanism, the system dynamically locks the highest abnormal peak value within the current trajectory during the streaming process, until the physical command of the current operation (Task Cycle) ends.

[0040] Overcoming the "abnormal dilution" effect and accurately capturing transient faults: In the actual operation of industrial robots, many severe mechanical or electrical faults often exhibit extremely short transient bursts. If a conventional direct mean strategy is used, these instantaneously spiked abnormally high scores are "diluted" and masked by the subsequent long period of normal low scores. The streaming maximum value extraction strategy in this step, once capturing a highly destructive transient abnormal peak, locks it in, thereby significantly improving the system's sensitivity to detecting sudden and transient faults.

[0041] Extremely low memory overhead and time complexity: thanks to the state transition equation With this design, the system does not need to cache the entire sequence in memory indefinitely. Only one needs to be maintained. A scalar comparison of space complexity is performed. This underlying design further demonstrates the engineering feasibility of this invention in achieving millisecond-level real-time response in edge computing gateways with extremely limited computing resources.

[0042] S5: Compare the trajectory-level anomaly score with the preset alarm threshold and output an anomaly warning signal.

[0043] After obtaining the score sequence after causal smoothing Then, the anomaly decision module performs the Causal-Max aggregation extraction operation.

[0044] Specifically, at any streaming truncation time, the system extracts the global maximum value from the generated smooth sequence:

[0045] This scalar As the final evaluation index characterizing the "abnormal intensity" of the current entire operating trajectory of an industrial robot (i.e., trajectory-level abnormality score).

[0046] Subsequently, the system reads the empirical alarm thresholds stored in the device's non-volatile memory. Based on simple Boolean logic judgment: If The system was determined to be malfunctioning.

[0047] Once an anomaly is triggered, the underlying control system will immediately issue a hard real-time interrupt command. Specific control responses include, but are not limited to: emergency braking of the robotic arm's servo drive via the programmable logic controller (PLC) (cutting off the enable signal), pushing a network alarm message containing an anomaly timestamp and anomaly channel feature vector to the back-end server of the manufacturing execution system (MES) in the plant, and triggering the on-site audible and visual alarms.

[0048] Furthermore, in a preferred embodiment, the preset alarm threshold is not solely determined manually, but can be automatically adjusted based on historical nominal data using unsupervised or semi-supervised algorithms. For example, during the calibration phase, the system can collect historical score distributions of equipment operating without abnormalities and use kernel density estimation (KDE) or extreme value theory (EVT) to calculate the extreme quantiles of the distribution (such as the 99.9th percentile). This adaptively generates and updates the alarm threshold, further enhancing the system's plug-and-play capability in different industrial environments.

[0049] A perfect balance between causal smoothing and maximum extraction: Simply using the direct maximum value is extremely sensitive to noise and easily generates a large number of false alarms; simply using the average value will mask instantaneous faults. This embodiment organically combines "causal smoothing" and "maximum value extraction" to achieve a reliable mapping from local multi-frequency evidence to global stable alarms.

[0050] Industrial Implementation Closed Loop: Step S5 transforms the abstract mathematical probability (NLL score) into a concrete physical blocking action (cutting off servo and PLC linkage), enabling this invention to move beyond the realm of pure algorithms and become a patented entity that effectively protects industrial safety.

Claims

1. An industrial robot stream anomaly detection method based on dynamic graph enhanced normalized flow, characterized by, Includes the following steps: S1: Continuously acquire multi-channel sensor time series during the operation of the industrial robot, and use a sliding window of preset length to divide the multi-channel sensor time series online to obtain the local window feature tensor of the current time step as streaming input data; S2: Input the streaming input data into a pre-trained dynamic graph augmented normalized flow model; wherein, the coupled sub-network of the dynamic graph augmented normalized flow model includes a dynamic graph augmentation module, which adaptively constructs a soft adjacency matrix representing the cross-sensor coupling strength based on the channel correlation within the local window feature tensor of the current time step, and jointly extracts local temporal patterns and dynamic cross-sensor dependency features during the reversible transformation process, and outputs the window-level anomaly score corresponding to the current time step; S3: Cache the window-level anomaly scores output at the current time step and historical time steps, and perform causal smoothing on the window-level anomaly score sequence sorted by time using a fixed-length causal smoothing window to suppress high-frequency fluctuations; S4: Extract the maximum value from the window-level anomaly score sequence after causal smoothing, and use it as the trajectory-level anomaly score of the industrial robot under the current trajectory; S5: Compare the trajectory-level anomaly score with a preset alarm threshold. If the trajectory-level anomaly score is greater than the alarm threshold, output an abnormal state warning signal.

2. The industrial robot streaming anomaly detection method of claim 1, wherein, In step S1, the online division of the multi-channel sensor time series using a sliding window of a preset length specifically includes: During continuous operation of the industrial robot, the time series of the multi-channel sensors arriving continuously are received in a streaming manner. In the current inference step, there is no need to wait for the complete operation trajectory of the industrial robot to end; only a fixed-length data segment within a preset time interval is extracted as the local window feature tensor. This ensures that the memory usage and single-step computation of the streaming input data during the inference process are independent of the total length of the industrial robot's operating trajectory, limiting the space complexity to a constant level related to the sliding window length, and avoiding hardware resource overload caused by full trajectory caching.

3. The industrial robot stream anomaly detection method of claim 1, wherein, In S2, the process of adaptively constructing the soft adjacency matrix characterizing the cross-sensor coupling strength specifically includes: Calculate the inner product of the local window feature tensor and its transpose matrix; Divide the inner product by the square root of the preset length of the sliding window to obtain the scaled correlation matrix; The soft adjacency matrix is ​​generated by applying a normalized exponential function (Softmax) along a specific dimension of the correlation matrix.

4. The industrial robot stream anomaly detection method according to claim 1 or 2, characterized in that, The dynamic graph enhancement module includes: Parallel temporal convolutional branches and graph propagation branches; The temporal convolution branch extracts local temporal patterns from the streaming input data through one-dimensional convolution. The graph propagation branch utilizes the soft adjacency matrix to perform cross-channel graph propagation in order to capture dynamic cross-sensor dependent features; The outputs of the temporal convolution branch and the graph propagation branch are fused through a residual structure and then input into the subsequent layers of the dynamic graph-enhanced normalized flow model.

5. The method according to claim 1, characterized in that, In step S3, the step of performing causal smoothing on the time-ordered window-level outlier score sequence using a fixed-length causal smoothing window specifically includes: Define the index of the current time step as Within the window, sum the window-level anomaly score sequence and divide the sum by . The smoothed window-level anomaly score is obtained.

6. The method according to claim 1, characterized in that, The training process of the dynamic graph-enhanced normalized flow model includes: The multi-channel sensor time series of the industrial robot under nominal normal operating conditions are obtained as a training set containing only positive samples; Based on the maximum likelihood estimation principle, the parameters of the dynamic graph-enhanced normalized flow model are optimized by minimizing the negative log-likelihood (NLL) loss function output by the dynamic graph-enhanced normalized flow model.

7. A flow cytometry anomaly detection device for industrial robots, characterized in that, include: Streaming input building module: Used to continuously acquire multi-channel sensor time series during the operation of industrial robots, and to perform online partitioning using a sliding window of preset length to obtain the local window feature tensor of the current time step; Dynamic graph augmented normalized flow module: It is used to input the local window feature tensor into the pre-trained dynamic graph augmented normalized flow model, and output the window-level anomaly score corresponding to the current time step by constructing a soft adjacency matrix that characterizes the cross-sensor coupling strength. Causal smoothing module: Used to perform causal smoothing on the window-level outlier score sequence sorted by time using a fixed-length causal smoothing window; Anomaly Decision Module: Used to extract the maximum value of the smoothed window-level anomaly score sequence as the trajectory-level anomaly score, and output an anomaly warning signal according to a preset alarm threshold.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing the computer program, implements the steps of the industrial robot streaming anomaly detection method based on dynamic graph enhanced normalized flow as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the industrial robot flow anomaly detection method based on dynamic graph enhanced normalized flow as described in any one of claims 1-5.

Images