Hydropower station generator rotor hoisting monitoring system based on multi-modal data fusion

Abstract

The invention provides a hydropower station generator rotor hoisting monitoring system based on multi-modal data fusion, and relates to the technical field of hydropower stations. The hydropower station generator rotor hoisting monitoring system based on multi-modal data fusion comprises an intelligent induction layer, the intelligent induction layer is connected with a collaborative decision-making layer, and the collaborative decision-making layer is connected with a dynamic execution layer; the intelligent sensing layer collects and processes multi-modal data in the hoisting process in real time through a task driving sensing module, an edge calculation optimization module and a multi-modal data fusion module. According to the invention, on the aspect of intelligent perception, on-demand adjustment of acquisition strategies, data preprocessing and fusion of multi-source information, collaborative decision making, dynamic path planning, decision transparency guarantee, risk early warning in advance, self-adaptive adjustment of operation during dynamic execution, three-dimensional interaction and unified scheduling of equipment are realized, so that the hoisting quality, efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hydropower stations, and in particular to a hydropower station generator rotor hoisting monitoring system based on multimodal data fusion. Background Art

[0002] A hydropower station is a comprehensive engineering facility that converts water energy into electricity. It typically consists of a reservoir formed by retaining and discharge structures, a water diversion system, a power plant, and electromechanical equipment. High-water levels in the reservoir flow through the diversion system into the power plant, driving the turbine generator sets to generate electricity. This electricity is then fed into the power grid via step-up transformers, switchyards, and transmission lines. Inspection of the generator rotor during installation is crucial for ensuring installation accuracy, ensuring equipment integrity, ensuring safe operation, and adhering to quality standards. As the core component of the generator, the rotor requires extremely high installation position and angular accuracy. Inspection ensures the accurate relative position of the rotor and stator, preventing uneven air gaps caused by installation deviations that could affect power generation efficiency and power quality. Rotor components can be damaged during installation due to impact, vibration, and other factors. Inspection can promptly identify potential damage, such as winding damage, loose poles, and loose or missing bolts, allowing for prompt repair or treatment to prevent further malfunctions caused by damaged equipment. During generator operation, the rotor spins at high speed. If hidden dangers are not promptly discovered and eliminated, they can cause serious safety accidents during operation, threatening the safety of equipment and personnel. Inspection can nip safety hazards in the bud. Inspection is also a key component of construction quality control. By evaluating and verifying various inspection indicators, we ensure that the quality of rotor hoisting meets relevant standards and regulations, laying the foundation for the reliable operation of the entire hydropower station.

[0003] Existing hydropower station rotor hoisting monitoring technologies suffer from common flaws, including rigid data collection, delayed processing, a single decision-making mechanism, insufficient execution flexibility, and weak multi-device coordination. Regarding data collection, traditional systems rely on fixed, single-type sensors (such as vibration sensors or cameras). Their parameters and sampling strategies cannot be dynamically adjusted according to the hoisting task, resulting in missing or redundant data during critical stages such as lifting and translation. For example, rotor posture is not monitored specifically during lifting, and obstacles are not tracked in real time during translation. This results in incomplete and poorly targeted data coverage. Regarding data processing, a centralized approach requires all raw data to be transmitted to the cloud or a central server, resulting in high latency and bandwidth pressure, making it impossible to meet the real-time requirements of hoisting. In complex environments, the transmission delay of multimodal data, such as image and mechanical data, can exceed safety thresholds, increasing operational risks. Decision-making mechanisms rely on manual experience or static path planning algorithms, which cannot dynamically adapt to environmental changes such as temporary obstacles and equipment anomalies. This results in low path planning efficiency and poor fault tolerance. Furthermore, collaborative decision-making among multiple devices lacks transparency and consistency, leading to frequent cross-system command conflicts. The execution layer uses preset parameters (such as fixed hoisting speeds and attitude thresholds), making it impossible to dynamically adjust operations based on real-time feedback. Sudden situations such as sudden wind speed changes and equipment failures require manual intervention, resulting in slow response and a high risk of chain reactions. Human-machine interaction relies on a two-dimensional monitoring interface, making it difficult for operators to intuitively perceive the rotor trajectory and obstacle distribution in three dimensions, leading to inefficient decision-making. There is a lack of a unified mechanism for multi-device coordination. Cranes, sensors, and monitoring systems use independent communication protocols, resulting in high data exchange latency and asynchronous commands, hindering the smoothness of hoisting operations and even causing safety incidents due to command conflicts. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a hydropower station generator rotor hoisting monitoring system based on multimodal data fusion, which solves the problems of rigid data collection, delayed processing, single decision-making mechanism and insufficient flexibility of the execution layer in the existing technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a hydropower station generator rotor hoisting monitoring system based on multimodal data fusion, comprising an intelligent sensing layer, the intelligent sensing layer is connected to a collaborative decision-making layer, and the collaborative decision-making layer is connected to a dynamic execution layer;

[0006] The intelligent perception layer collects and processes multimodal data in the lifting process in real time through the task-driven perception module, edge computing optimization module and multimodal data fusion module. The collaborative decision-making layer generates lifting path planning and risk response strategies based on the swarm intelligence algorithm module, distributed decision-making mechanism module and dynamic risk assessment module model. The dynamic execution layer performs lifting operations and adjusts parameters in real time through adaptive control, mixed reality interaction and multi-device linkage technology. The intelligent perception layer, collaborative decision-making layer and dynamic execution layer form a closed-loop control system through data interaction and command transmission.

[0007] Preferably, the task-driven perception module dynamically adjusts the sensor deployment position and sampling strategy according to the hoisting stage, the edge computing optimization module deploys a lightweight model at the sensor end to process data in real time and reduce transmission delay, and the multimodal data fusion module eliminates redundancy and noise in multi-sensor data through an adaptive weighted fusion algorithm.

[0008] Preferably, the task-driven perception module optimizes sensor parameters based on a dynamic programming algorithm, specifically including:

[0009] During the lifting phase, priority is given to collecting rotor attitude and stability data, while during the translation phase, priority is given to collecting path planning and obstacle monitoring data.

[0010] Preferably, the edge computing optimization module includes a lightweight convolutional neural network model for real-time feature extraction and anomaly detection of image data.

[0011] Preferably, the multimodal data fusion module adopts an extended Kalman filter algorithm to dynamically adjust the fusion weight according to the confidence of the sensor data.

[0012] Preferably, the swarm intelligence algorithm module generates obstacle avoidance paths and optimal lifting trajectories based on the ant colony optimization algorithm or the particle swarm optimization algorithm, the distributed decision-making mechanism module realizes multi-device collaborative decision-making and records the decision-making process through blockchain technology, and the dynamic risk assessment module calculates risk probabilities in real time based on the Bayesian network model and generates fault-tolerant instructions.

[0013] Preferably, the distributed decision-making mechanism module automatically executes the decision-making plan through smart contracts and uses blockchain nodes to verify the consistency of multi-device decisions.

[0014] Preferably, the dynamic risk assessment module updates environmental parameters and equipment status data in real time through a Bayesian network, predicts collision risks and triggers early warning signals.

[0015] Preferably, the adaptive control module adjusts the hoisting speed and rotor spatial attitude in real time based on the model predictive control algorithm, the mixed reality interaction module superimposes the virtual hoisting trajectory and obstacle information onto the real-time picture of the actual scene, and the multi-device linkage control module realizes the command synchronization of the crane, sensor and monitoring system through software-defined network technology.

[0016] Preferably, the multi-device linkage control module dynamically allocates network resources based on the open flow protocol to ensure real-time transmission of lifting instructions and millisecond-level response of equipment collaborative operations.

[0017] The present invention provides a hydropower station generator rotor hoisting monitoring system based on multimodal data fusion. It has the following beneficial effects:

[0018] The present invention provides a hydropower station generator rotor hoisting monitoring system based on multimodal data fusion. In terms of intelligent perception, the present invention dynamically adjusts sensor deployment and sampling strategies according to the hoisting stage based on a dynamic programming algorithm, collects key data in a targeted manner, deploys a lightweight model on the sensor side to pre-process multimodal data, reduces transmission volume, achieves rapid response, fuses multi-sensor data, and eliminates noise interference. In terms of collaborative decision-making, an ant colony optimization algorithm is used to generate a dynamic obstacle avoidance path to adapt to environmental changes. Blockchain technology is used to achieve multi-device collaborative decision-making to ensure transparent decision-making. Risks are assessed and warned in real time based on a Bayesian network model. In terms of dynamic execution efficiency, an algorithm is used to adjust the hoisting speed and posture in real time to adapt to emergencies. A three-dimensional visualization interface is provided through a mixed reality device. Equipment is uniformly dispatched based on software-defined network technology to ensure command synchronization. The system improves hoisting efficiency, safety, and fault tolerance through a "perception-decision-execution" closed-loop architecture, providing an innovative solution for hydropower station equipment operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the system framework flow of the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] like Figure 1 As shown, an embodiment of the present invention provides a hydropower station generator rotor hoisting monitoring system based on multimodal data fusion, and the specific implementation method is as follows:

[0022] Intelligent perception layer implementation

[0023] Task-driven perception module

[0024] Step 1: Prioritize tasks according to the lifting phase (lifting, translation, placement), for example:

[0025] During the hoisting phase, the tilt sensor and laser rangefinder are deployed to collect real-time data on the rotor's attitude (tilt angle, height above the ground) and hook force. The sampling frequency is set to 100 Hz. During the translation phase, the binocular camera and LiDAR sensor are activated to scan obstacles along the hoisting path and generate a 3D point cloud map. The sampling frequency is adjusted to 50 Hz.

[0026] Step 2: Optimize sensor deployment through dynamic programming algorithms. For example, prioritize edge sensors in confined environments to expand coverage. Develop a task scheduler based on ROS (Robot Operating System) to dynamically allocate sensor resources and adjust parameters.

[0027] Edge computing optimization module

[0028] Step 1: Embed an edge computing unit (NVIDIA Jetson Nano) in the sensor nodes (camera, vibration sensor) and deploy a lightweight CNN model (MobileNetV3).

[0029] Step 2: Process the data stream in real time:

[0030] Image data: CNN is used to extract rotor edge features and abnormal vibration areas, compressing the data to 10% of its original size. Mechanical data: Fast Fourier transform (FFT) is used to analyze the force spectrum of the lifting ropes to detect potential breakage risks. The TensorFlow Lite framework is used to optimize model inference speed, achieving a single-frame image processing time of ≤20ms.

[0031] Multimodal data fusion module

[0032] Step 1: Normalize the data from different sensors (coordinate alignment and timestamp synchronization).

[0033] Step 2: Use the Extended Kalman Filter (EKF) to fuse multi-source data: input LiDAR point cloud (position), tilt sensor (attitude), and camera (visual features). Output: high-precision rotor 3D pose (error ≤ ±1 cm).

[0034] Step 3: Dynamically adjust data weights through an adaptive weighting algorithm, for example:

[0035] When the camera is disturbed by light, the weight of visual data is reduced and the weight of LiDAR data is increased to 80%.

[0036] Collaborative decision-making implementation

[0037] Swarm intelligence algorithm module

[0038] Step 1: Initialize the parameters of the ant colony optimization algorithm (number of ants = 50, pheromone volatility coefficient = 0.1) and model the hoisting environment as a grid map.

[0039] Step 2: Iterate to generate paths:

[0040] Ants simulate path exploration, avoid obstacle grids (marked as impassable), and accumulate the optimal solution with the shortest path and the fewest turns.

[0041] Step 3: Output the optimal path to the crane control system, which updates the path in real time (replanning when temporary obstacles are detected). The algorithm is implemented using the Python DEAP framework, with a planning time of ≤5 seconds.

[0042] Distributed decision-making mechanism module

[0043] Step 1: Build a private blockchain network and deploy Hyperledger Fabric nodes (the crane, sensor, and monitoring system are each independent nodes).

[0044] Step 2: Define decision rules through smart contracts (Chaincode):

[0045] The crane node proposes a path → the sensor node verifies the environmental safety → the monitoring node confirms → the path is executed after the majority of nodes agree.

[0046] Step 3: Record the decision log to the blockchain to ensure that the operation is traceable and tamper-proof.

[0047] Dynamic risk assessment module

[0048] Step 1: Construct a Bayesian network topology, where nodes include ambient wind speed, rotor offset, equipment health status, etc.

[0049] Step 2: Input sensor data in real time and calculate risk probability:

[0050] If the wind speed is >10m / s and the offset is >5cm, the collision risk probability increases to 85%, triggering an emergency braking command.

[0051] Step 3: Generate a fault-tolerant solution (switch to an alternative lifting path or reduce the lifting speed).

[0052] Dynamic Execution Layer Implementation

[0053] Adaptive Control Module

[0054] Step 1: Establish a lifting dynamics model and define state variables (speed, attitude angle) and control variables (motor torque, hydraulic pressure).

[0055] Step 2: Use the model predictive control (MPC) algorithm:

[0056] Predict the hoisting trajectory within the next three seconds and optimize control commands to ensure rotor attitude error ≤±2cm. Generate C code based on MATLAB / Simulink and deploy it to the crane's PLC controller.

[0057] Mixed reality interaction module

[0058] Step 1: Capture the operator's field of view through Microsoft HoloLens 2 and overlay virtual information:

[0059] Green trajectory line: shows the planned lifting path. Red heat map: marks high-risk obstacle areas.

[0060] Step 2: Support gesture interaction, such as swiping your palm to change the perspective and using voice commands to pause the rig. Develop MR applications using the Unity3D engine, with rendering latency ≤ 50ms.

[0061] Multi-device linkage control module

[0062] Step 1: Build an SDN network based on the OpenFlow protocol to uniformly manage the communications of cranes, sensors, and monitoring systems.

[0063] Step 2: Dynamically allocate network bandwidth:

[0064] Crane control commands have the highest priority (60% of bandwidth), ensuring that command transmission delay is ≤10ms. Sensor data takes the second place (30% of bandwidth), and surveillance video traffic has the lowest priority (10%).

[0065] Step 3: Use the SDN controller to monitor the device status in real time and automatically switch to the backup link when a failure occurs.

[0066] This technology uses a closed-loop "perception-decision-execution" architecture, combined with innovative methods such as task-driven perception, swarm intelligence algorithms, and adaptive control, to systematically address the data rigidity, decision-making lag, and execution redundancy issues of traditional lifting monitoring systems. It achieves high-precision, high-real-time, and high-safety intelligent lifting operations, and provides a standardized technical paradigm for the operation and maintenance of large-scale equipment in hydropower stations.

[0067] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A hydropower station generator rotor installation monitoring system based on multimodal data fusion, including an intelligent sensing layer, characterized by: The intelligent sensing layer is connected to a collaborative decision-making layer, and the collaborative decision-making layer is connected to a dynamic execution layer; The intelligent perception layer collects and processes multimodal data in the lifting process in real time through the task-driven perception module, edge computing optimization module and multimodal data fusion module. The collaborative decision-making layer generates lifting path planning and risk response strategies based on the swarm intelligence algorithm module, distributed decision-making mechanism module and dynamic risk assessment module model. The dynamic execution layer performs lifting operations and adjusts parameters in real time through adaptive control, mixed reality interaction and multi-device linkage technology. The intelligent perception layer, collaborative decision-making layer and dynamic execution layer form a closed-loop control system through data interaction and command transmission.

2. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1 is characterized by: The task-driven perception module dynamically adjusts the sensor deployment position and sampling strategy according to the hoisting stage. The edge computing optimization module deploys a lightweight model on the sensor side to process data in real time and reduce transmission delay. The multimodal data fusion module eliminates redundancy and noise in multi-sensor data through an adaptive weighted fusion algorithm.

3. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1 is characterized by: The task-driven perception module optimizes sensor parameters based on a dynamic programming algorithm, specifically including: During the lifting phase, priority is given to collecting rotor attitude and stability data, while during the translation phase, priority is given to collecting path planning and obstacle monitoring data.

4. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1 is characterized by: The edge computing optimization module includes a lightweight convolutional neural network model for real-time feature extraction and anomaly detection of image data.

5. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1, characterized in that: The multimodal data fusion module adopts the extended Kalman filter algorithm to dynamically adjust the fusion weight according to the confidence of the sensor data.

6. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1, characterized in that: The swarm intelligence algorithm module generates obstacle avoidance paths and optimal lifting trajectories based on the ant colony optimization algorithm or the particle swarm optimization algorithm. The distributed decision-making mechanism module realizes multi-device collaborative decision-making and records the decision-making process through blockchain technology. The dynamic risk assessment module calculates risk probabilities in real time and generates fault-tolerant instructions based on the Bayesian network model.

7. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1, characterized in that: The distributed decision-making mechanism module automatically executes decision-making plans through smart contracts and uses blockchain nodes to verify the consistency of multi-device decisions.

8. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1, characterized in that: The dynamic risk assessment module updates environmental parameters and equipment status data in real time through a Bayesian network, predicts collision risks and triggers early warning signals.

9. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1, characterized in that: The adaptive control module adjusts the hoisting speed and rotor spatial attitude in real time based on the model predictive control algorithm. The mixed reality interaction module superimposes the virtual hoisting trajectory and obstacle information on the real-time image of the actual scene. The multi-device linkage control module synchronizes the instructions of the crane, sensors and monitoring system through software-defined network technology.

10. The hydropower station generator rotor installation monitoring system based on multimodal data fusion according to claim 1, characterized in that: The multi-device linkage control module dynamically allocates network resources based on the open flow protocol to ensure real-time transmission of lifting instructions and millisecond-level response of equipment collaborative operations.

Images