Distributed edge video vibration monitoring system and method thereof

Abstract

The invention discloses a distributed edge video vibration monitoring system which is suitable for non-contact vibration detection of industrial sites such as a transformer substation. The platform is composed of a plurality of RDKX3 edge intelligent nodes containing cameras, a terminal server and a Web-based man-machine interaction system, and the star topology is adopted for communication in the same local area network. The edge nodes are responsible for lightweight preprocessing and video transmission through Wi-Fi; a terminal server carries out data caching and preprocessing in a centralized manner, and a scheduler dynamically allocates tasks among a main GPU / a second GPU / a CPU / edge local reasoning, so that multi-path parallel and load balancing are realized while real-time performance is ensured; the deep learning reasoning module estimates the oscillation component, the output frequency and the amplitude of the video sequence of the selected ROI. According to the invention, high-reliability, multi-node and low-delay vibration monitoring can be realized without contacting the tested equipment, and the system has a multi-level architecture which can be expanded to a'regional server cloud platform ', is suitable for engineering-scale deployment, and has good universality and maintainability.

Description

Technical Field

[0001] This invention relates to the field of power equipment condition monitoring and machine vision technology, specifically to a human-computer interactive video vibration monitoring platform based on distributed edge acquisition and centralized deep learning inference, which is suitable for non-contact vibration detection, alarm and visualization in industrial sites such as substations. Background Technology

[0002] Traditional vibration monitoring of power equipment relies heavily on contact sensors (such as accelerometers), requiring power outages for installation or long-term wiring, resulting in high construction and maintenance costs. Furthermore, point-based sensors struggle to cover multiple vibration sources on complex equipment. In recent years, video vibrometry and video amplification technologies have developed rapidly. Combined with deep learning-based spatiotemporal feature modeling, they can estimate the frequency and amplitude of minute oscillations in equipment from video sequences without physical contact with the target. However, industrial environments present challenges such as multiple cameras, multiple camera positions, complex networks, varying illumination, and occlusion. As the number of cameras increases, centralized processing by a single machine easily encounters computational bottlenecks and latency jitter, making it difficult to meet the engineering requirements of "multi-source concurrency + real-time alarm."

[0003] Therefore, there is an urgent need for a system that combines distributed edge acquisition, centralized inference over a local area network, and a closed-loop front-end interaction: performing only lightweight preprocessing at the edge to reduce bandwidth and jitter, centralizing high-performance deep learning inference on the terminal server, and dynamically allocating resources among multiple GPUs / CPUs / edges through scheduling and load balancing to achieve a low-latency, highly scalable, and maintainable engineering solution. Summary of the Invention

[0004] A distributed edge video vibration monitoring system, comprising: a) Multiple edge intelligent nodes, each containing an industrial camera and an RDKX3 processing module, for real-time video acquisition and lightweight preprocessing near the device under test; b) A terminal server, located on the same local area network as the plurality of edge intelligent nodes, communicating directly with the plurality of edge intelligent nodes using a star topology; the terminal server includes: Access gateway and buffer queue are used to receive video streams transmitted via Wi-Fi and perform buffering and time synchronization; The data preprocessing pipeline is used to perform frame correction, brightness normalization, and ROI mapping on video frames. The scheduler is used to dynamically allocate tasks among various computing resources based on queue depth and hardware load. It prioritizes assigning tasks to the main GPU, assigns them to the second GPU when the main GPU is busy, switches to the CPU for emergency processing when GPU resources are saturated, and can also distribute tasks to edge intelligent nodes for local inference in advanced deployments. The deep learning inference module is used to estimate the video oscillation components based on the ROI and output the frequency and amplitude. The inference engine is GPU accelerated. The results evaluation module is used to robustly process the inference results and generate alarm flags or levels; The message push module is used to push results to the front end via WebSocket using a unified JSON interface; c) Human-Computer Interaction System (HMI) is used for multi-camera preview, ROI visualization editing, parameter and threshold configuration, real-time feedback, log and health monitoring, permission management, and to present and alert on the received JSON results.

[0005] The monitoring method of this system at the substation site is characterized by including: S1: Multiple edge intelligent nodes access the same local area network via Wi-Fi and send video streams or clips to the terminal server using one of RTSP / WebRTC / TCP. S2: The terminal server completes access registration, cache queuing and time synchronization, and sends the video frames into the data preprocessing pipeline; S3: The scheduler performs dynamic scheduling based on the utilization of the main GPU, the utilization of the second GPU, the CPU idle time and the queue length, selects available resources among the cascaded paths of "main GPU → second GPU → CPU → edge local inference", and creates parallel processing threads as needed; S4: The deep learning inference module performs video oscillation estimation on the ROI to obtain the frequency and amplitude; S5: The results evaluation module generates alarm flags or levels; S6: The message push module pushes messages to the HMI via WebSocket using a unified JSON (containing at least node_id, timestamp, frequency, amplitude, and alert). The HMI updates the interface and highlights alerts when they are triggered.

[0006] The timestamp in the JSON uses the UTC standard time format and is accurate to milliseconds. The amplitude supports standardized values ​​or is converted to millimeters after calibration. The HMI supports historical query and export.

[0007] The network topology can be expanded into a multi-level structure, introducing regional servers to aggregate monitoring results from multiple substations, and then managed and optimized globally by the upper-level cloud platform.

[0008] The HMI supports online ROI polygon drawing, hot updating of threshold and filtering parameters, role permissions and auditing, and the server supports image deployment and OTA upgrades.

[0009] A computer-readable storage medium storing instructions that, when executed by a processor, enable the implementation of the method of claim 2.

[0010] The beneficial effects of this invention are: it enables multi-node concurrent, low-latency, and highly reliable video vibration monitoring without changing the field equipment or relying on contact sensing; through the cascaded scheduling of "main GPU → second GPU → CPU → edge", it ensures stable output results even under load fluctuations and fault conditions; the unified JSON interface and front-end HMI enable the system to have good visualization, maintainability, and scalability, making it suitable for large-scale deployment in substations and other scenarios. Attached Figure Description

[0011] Figure 1 Schematic diagram of the overall architecture and multi-terminal presentation protocol of the distributed edge video vibration monitoring system.

[0012] Figure 2 Data processing and scheduling flowchart (caching → preprocessing → scheduling → GPU / CPU / edge fallback → inference → evaluation → output).

[0013] Figure 3 : Schematic diagram of computing service startup and branch parallel scheduling process (configuration reading, main scheduling, branch creation and parallel execution).

[0014] Figure 4 : A schematic diagram of the structure of the WebSocket JSON monitoring result message pushed by the terminal server to the HMI.

[0015] Figure 5 : Diagram of on-site deployment topology and expansion options (single-site star topology, regional server, cloud platform delivery). Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0017] This invention provides a distributed edge video vibration monitoring system, which consists of multiple RDKX3 edge nodes connected to cameras, a terminal server in the same local area network, and a web-based human-computer interaction (HMI). The system constructs a real-time vibration monitoring closed loop around a star topology and centralized deep learning inference.

[0018] Step 1: Deploy several edge intelligent nodes (RDKX3 + industrial cameras) in the target substation. Each node is installed near the device under test and connected to the same local area network via Wi-Fi. All nodes communicate directly with the terminal server in a star topology.

[0019] Step 2: The edge node performs lightweight preprocessing on the video stream (basic shaking, compression, timestamp annotation, optional ROI cropping) and sends the stream / segment to the access gateway of the terminal server via RTSP / WebRTC / TCP; the node retains a minimal state for subsequent expansion of local emergency inference.

[0020] Step 3: The terminal server maintains the access registry and cache queue, buffers and synchronizes the time of each video stream, and uniformly transfers it to the data preprocessing pipeline (frame correction, brightness normalization, ROI mapping) to provide stable input for deep learning inference.

[0021] Step 4: The server's built-in scheduler dynamically allocates resources based on the number of paths, frame rate, queue length, and hardware load: priority is given to the main GPU; if the main GPU is busy, the load is distributed to the second GPU; if GPU resources are saturated, the load is switched to the CPU emergency path; in advanced deployments, it can be distributed to edge nodes for local inference as a supplement, thereby achieving parallelism and load balancing among multiple resources.

[0022] Step 5: In the deep learning inference module, spatiotemporal feature modeling and oscillation component estimation are performed on the video sequence of the selected ROI, and the frequency (Hz) and amplitude (normalized value or calibrated and converted to millimeters) are output. The inference engine is accelerated by TensorRT / GPU and low end-to-end latency is guaranteed by multi-threading / asynchronous queue.

[0023] Step 6: In the result evaluation module, perform robustness processing on the model output (moving median / anomaly suppression / confidence screening), generate alarm flags or levels in combination with site threshold strategies, and complete multi-source aggregation (fusion of multiple camera positions on the same device).

[0024] Step 7: The server pushes a unified JSON message to the HMI in real time via WebSocket: node_id, timestamp (UTC, ms), frequency (Hz), amplitude (normalized or mm), and alert (boolean or level); the front end updates the curves and status according to the node dimension and supports historical playback and export.

[0025] Step 8: The HMI is built using a stable and mature Web technology stack, providing camera / node management, ROI visualization editing, parameter and threshold configuration, real-time preview and result display, log and health monitoring, permission and multi-role management, etc.; all configuration items are persistent and can be hot-updated online.

[0026] Step 9: The system supports elastic scaling: Add edge nodes and wireless coverage as needed in a star topology; or introduce regional servers to form a multi-level architecture (local servers within the site + aggregation on the upper-level cloud platform). Through mirrored deployment, centralized log / metric collection, and security hardening (TLS, authentication, auditing), maintainable project implementation is achieved.

[0027] Example A: Star topology deployment of a single server within the site (corresponding to...) Figure 1 , Figure 2 ) In a substation, N edge nodes (RDKX3 + industrial cameras) are deployed, all accessing the substation's Wi-Fi via the same SSID. Each node transmits H.264 / H.265 encoded video streams to a terminal server via RTSP or WebRTC. The server's network port is connected to the operator workstation (HMI) via a wired connection. Dashed lines represent wireless links, and solid lines represent wired links. Nodes and access points can be added to increase coverage as needed, or a two- or three-tiered structure of "regional server → cloud platform" can be introduced to aggregate cross-site data. Figure 2 ).

[0028] Example B: Data Processing and Scheduling (corresponding) Figure 3 ) The server maintains a buffer queue and input registry for each stream at the access layer; the preprocessing module performs frame correction, ROI mapping, and intensity normalization; the scheduler periodically scans the queue depth and GPU / CPU utilization, allocating tasks using a strategy of "primary GPU priority, secondary GPU offloading, CPU emergency response, and edge fallback," and dynamically creates processing threads for each branch computation to ensure a smooth pipeline. The inference module uses a preloaded deep learning model to estimate the dominant frequency and amplitude based on ROI and outputs confidence scores. The evaluation module performs jitter suppression and thresholding strategies, generating alerts or ratings. The end-to-end target latency is less than a predetermined threshold (e.g., 100–200ms, depending on resolution and stream configuration).

[0029] Example C: Data Interface Protocol and HMI (corresponding) Figure 4 , Figure 5 ) The server pushes unified JSON via WebSocket: { "node_id":"NODE_001", "timestamp":"2025-07-04T20:45:30.120Z", "frequency":50.0, "amplitude":0.02, "alert":true } The front-end updates the frequency / amplitude curves of the corresponding nodes accordingly and issues a highlight / pop-up alarm when alert=true. The HMI provides ROI polygon drawing, threshold / filter parameter configuration, real-time FPS / latency / occupancy dashboard, alarm confirmation and export, and role / access control. Adding new fields (such as waveform segments and temperature) can be extended without changing the main workflow.

[0030] Example D: Reliability and Operations Supports node / model / frontend mirroring and OTA upgrades; centralized collection of logs and metrics (load, frame loss, latency); threshold-triggered self-healing (automatic resolution downgrading, channel switching, and CPU / edge emergency activation). TLS and authentication are enabled across the entire link; servers and HMIs support token-based access control and auditing.

[0031] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A distributed edge video vibration monitoring system, characterized in that, include: a) Multiple edge intelligent nodes, each containing an industrial camera and an RDKX3 processing module, for real-time video acquisition and lightweight preprocessing near the device under test; b) A terminal server, located in the same local area network as the multiple edge intelligent nodes, communicates directly with the multiple edge intelligent nodes using a star topology; The terminal server includes: Access gateway and buffer queue are used to receive video streams transmitted via Wi-Fi and perform buffering and time synchronization; The data preprocessing pipeline is used to perform frame correction, brightness normalization, and ROI mapping on video frames. The scheduler is used to dynamically allocate tasks among various computing resources based on queue depth and hardware load. It prioritizes assigning tasks to the main GPU, assigns them to the second GPU when the main GPU is busy, switches to the CPU for emergency processing when GPU resources are saturated, and can also distribute tasks to edge intelligent nodes for local inference in advanced deployments. The deep learning inference module is used to estimate the video oscillation components based on the ROI and output the frequency and amplitude. The results evaluation module is used to robustly process the inference results and generate alarm flags or levels; The message push module is used to push results to the front end via WebSocket using a unified JSON interface; c) Human-Computer Interaction System (HMI) is used for multi-camera preview, ROI visualization editing, parameter and threshold configuration, real-time feedback, log and health monitoring, permission management, and to present and alert on the received JSON results.

2. The system as described in claim 1, characterized in that, The lightweight preprocessing performed by the edge intelligent node includes at least basic shaking, compression, timestamp annotation, and optional ROI cropping, and sends the video stream or clips to the access gateway of the terminal server using one of RTSP / WebRTC / TCP.

3. The system as described in claim 1, characterized in that, The terminal server maintains the access registry and caches, queues, and synchronizes the time of each video stream before sending them to the data preprocessing pipeline for frame correction, brightness normalization, and ROI mapping.

4. The system as described in claim 1, characterized in that, The scheduler performs periodic scanning and dynamic scheduling based on the number of paths, frame rate, queue length, and GPU / CPU utilization, and creates parallel processing threads for branch computation as needed to achieve multi-path parallelism and load balancing.

5. The system as described in claim 1, characterized in that, The deep learning inference module uses TensorRT and GPU acceleration, and achieves end-to-end low-latency output of the frequency and amplitude through multi-threading and / or asynchronous queues.

6. The system as described in claim 1, characterized in that, The result evaluation module performs robustness processing on the model output. The robustness processing includes at least moving median, anomaly suppression and confidence screening, and generates alarm flags or alarm levels in combination with site threshold strategies. It also supports multi-source aggregation and fusion of multiple locations on the same device.

7. The system as described in claim 1, characterized in that, The unified JSON pushed by the message push module includes at least: node_id, timestamp, frequency, amplitude, and alert. The timestamp uses UTC and is accurate to milliseconds, the frequency is Hz, the amplitude is a normalized value or converted to millimeters after calibration, and the alert is a boolean or level.

8. The system as described in claim 1, characterized in that: (1) The system supports elastic expansion, allowing the addition of edge intelligent nodes and wireless coverage as needed in a star topology, and can introduce regional servers to form a multi-level architecture, which is then aggregated and managed by the upper-layer cloud platform; and / or (2) The system supports image deployment and OTA upgrade, and centrally collects logs and indicators to trigger self-healing based on thresholds. Self-healing includes at least automatic resolution reduction, channel switching and initiation of CPU emergency and / or edge emergency; and enables TLS and authentication across the entire link, and supports token-based access control and auditing for servers and HMI.

9. A method for on-site vibration monitoring in a substation using the system described in any one of claims 1-8, characterized in that, include: S1: Multiple edge intelligent nodes access the same local area network via Wi-Fi and send video streams or clips to the terminal server using one of RTSP / WebRTC / TCP. S2: The terminal server completes access registration, cache queuing and time synchronization, and sends the video frames into the data preprocessing pipeline; S3: The scheduler performs dynamic scheduling based on the utilization of the main GPU, the utilization of the second GPU, the CPU idle time and the queue length, selects available resources among the cascaded paths of "main GPU → second GPU → CPU → edge local inference", and creates parallel processing threads as needed; S4: The deep learning inference module performs video oscillation estimation on the ROI to obtain the frequency and amplitude; S5: The results evaluation module generates alarm flags or levels; S6: The message push module pushes messages to the HMI via WebSocket using a unified JSON format. The HMI then updates the interface and highlights alerts when alarms occur.

10. A computer-readable storage medium having instructions stored thereon, which, when executed by a processor, enable the implementation of the method of claim 9.

Images