Power plant intelligent maintenance method and system based on multi-modal dynamic graph learning

Abstract

The invention discloses a power plant intelligent maintenance method and system based on multi-modal dynamic graph learning. The method comprises the following steps: acquiring structured sensor data, unstructured data and equipment physical topology data of equipment operation in real time through a multi-source sensor cluster and an industrial terminal; the method comprises the following steps: preprocessing multi-modal data, and fusing multi-modal features by using a double-flow Transform architecture and a gated attention mechanism to generate a joint embedded representation; constructing a dynamic causal graph based on equipment physical topology data and sensor time sequence characteristics, updating an edge weight through a GraphSAGE algorithm, fusing domain rule constraints, and outputting equipment state information; generating a maintenance strategy through an improved near-end strategy optimization algorithm according to the state and the equipment health index; and finally, the maintenance strategy triggers third-level early warning of the DCS through an OPC UA protocol, and a maintenance instruction is accurately issued. According to the method, the defects of a traditional method in the aspects of data fusion, fault modeling and decision making are overcome, and the safety, the economical efficiency and the operation and maintenance intelligent level of power plant equipment are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent maintenance of power plant equipment, and in particular to a method and system for intelligent maintenance of power plants based on multimodal dynamic graph learning. Background Art

[0002] With the increasing complexity of the operating environment of power plant equipment and the continuous improvement of equipment reliability requirements, traditional equipment maintenance methods have exposed a series of problems that need to be solved urgently.

[0003] First, there are significant flaws in data fusion. Power plant equipment operation generates a large amount of multi-source data, including structured time series data and unstructured text data. However, traditional methods are unable to process this data. For example, the LSTM-ARIMA hybrid model proposed in some studies can only process structured time series data. It cannot interpret key semantic information in maintenance work orders, such as "abnormal bearing noise accompanied by abnormal lubricating oil temperature," resulting in a large number of valuable fault clues being overlooked. In terms of data fusion strategies, some early methods directly spliced multi-source data, which resulted in a sharp increase in the feature space dimensionality. When the measured feature dimension exceeded 5000, the model accuracy declined, seriously affecting the accuracy and effectiveness of subsequent data analysis.

[0004] Secondly, topological modeling has shortcomings. In actual operation, power plant equipment, especially equipment like steam turbines, experience dynamic coupling relationships under variable load conditions. However, existing technologies ignore these dynamic changes, resulting in high fault location errors. While static equipment relationship diagrams constructed by some conventional equipment relationship modeling methods can capture spatial correlations between devices, they lack explicit modeling of the causal propagation paths of faults, making it difficult to accurately trace the source of faults and unable to provide strong support for precise maintenance.

[0005] Furthermore, the lack of dynamic decision-making is a prominent issue. During actual power plant operation, external factors such as unit startup and shutdown, fluctuations in coal quality, and changes in ambient temperature and humidity all affect equipment status. For example, some existing maintenance strategies use fixed thresholds, which can generate false alarms during unit startup and shutdown, resulting in wasted maintenance resources. Some rule-based decision-making systems, however, are unable to respond to these external disturbances in real time and cannot adjust maintenance strategies based on the real-time status of equipment, significantly compromising the timeliness and effectiveness of maintenance decisions. Summary of the Invention

[0006] In order to solve the problems raised in the above background technology, the present invention provides a power plant intelligent maintenance method and system based on multimodal dynamic graph learning. The method and system realize the spatiotemporal feature alignment and dynamic fusion of multimodal data through a dual-stream Transformer architecture, construct a dynamic causal graph to analyze the equipment fault propagation path, and realize the adaptive optimization of maintenance strategies based on reinforcement learning, forming a complete closed loop of "data acquisition-feature fusion-causal modeling-intelligent decision-making-instruction execution", which effectively solves the core problems of traditional methods in insufficient multi-source data fusion capabilities, static topology modeling and lack of decision-making dynamics.

[0007] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0008] A method for intelligent maintenance of a power plant based on multimodal dynamic graph learning includes the following steps:

[0009] S1. Multi-source data acquisition: Through multi-source sensor clusters and industrial terminals, real-time acquisition of device operation structured sensor data, unstructured text data, and image data;

[0010] S2. Multimodal data fusion: First, preprocess the multimodal data to generate a joint time series feature vector of the sensor data. , text semantic feature vector and image space feature vector Then, through the dual-stream Transformer architecture cross-modal alignment network, the above features are aligned and fused with spatiotemporal features to generate a joint embedding representation containing spatiotemporal associations. ;

[0011] S3. Dynamic Causal Graph Construction: Based on the Joint Time Series Feature Vector of Device Physical Topology Data and Sensor Data , combined with text semantic feature vector Perform domain rule matching to build a dynamic causal graph consisting of a data layer, a feature extraction layer, and a causal reasoning layer. This identifies and outputs device status information, including each device's current failure probability, failure propagation path, root cause node information, and the strength of causal relationships between devices, for use in maintenance strategy generation.

[0012] S4. Reinforcement Learning Decision-Making: Joint Embedding Representation Generated by Fusion of Device Status Information and Multimodal Data Output from Dynamic Causal Graph , to calculate the equipment health index, and through the improved proximal policy optimization algorithm, the state vector containing the dynamic causal graph state and the three-dimensional reward function are used as input to generate maintenance strategies to achieve multi-objective optimization. The maintenance strategy types include routine inspection strategy, preventive maintenance strategy and emergency shutdown strategy;

[0013] S5. Maintenance plan generation and execution: Trigger the three-level warning mechanism based on the calculated equipment health index. Generate and execute corresponding specific instructions based on the specific maintenance strategy type generated in step S4. Different maintenance strategies correspond to specific actions in the action space.

[0014] Preferably, the multi-source sensor cluster in step S1 includes a vibration sensor, a temperature sensor, and a pressure sensor, and the sensor data includes vibration signals, temperature signals, and pressure signals. The vibration sensor collects vibration signals at a high frequency and can accurately capture subtle vibration changes during equipment operation. These changes are usually early signs of potential equipment failures; the temperature sensor uses a high-precision optical fiber sensor, which can perform full-length temperature monitoring of key equipment, providing a basis for thermal status assessment of the equipment; the pressure sensor collects boiler drum pressure in real time, providing important parameters for stability analysis of equipment operation.

[0015] Preferably, the industrial terminal in step S1 includes an industrial explosion-proof flat panel and an infrared camera. The text data is the maintenance log entered through the industrial explosion-proof flat panel, and the voice-to-text technology is used to generate structured fields to facilitate the subsequent rapid extraction of key information; the image data is collected by the infrared camera at a frequency of once every 30 minutes to obtain the surface temperature field distribution of the equipment, and the temperature difference is used to determine whether there is an abnormal heating area in the equipment.

[0016] Preferably, the data collected in step S1 also includes equipment physical topology data, which is a static connection relationship between devices constructed based on power plant design drawings and historical records, providing a basis for subsequent analysis of mutual influence between devices.

[0017] Preferably, the multimodal data in step S2 includes sensor data, text data, and image data, and a special preprocessing method is used for the multimodal data.

[0018] Preferably, when preprocessing the sensor data, a 1024-point fast Fourier transform is performed on the vibration signal to convert the time domain signal into frequency domain features. This conversion can highlight the frequency components of the signal, which is convenient for subsequent analysis of the characteristic frequency of the equipment vibration; local features are then extracted through a one-dimensional convolutional neural network, and the setting of its convolution kernel is optimized according to the characteristics of the equipment vibration to effectively capture the key changes in the vibration signal; then, the timing dependency is captured through a bidirectional long short-term memory network, which can learn the before and after correlation information of the vibration signal in the time series and accurately grasp the dynamic trend of the equipment vibration.

[0019] Preferably, the temperature signal and pressure signal in the sensor data are normalized, their value range is mapped to [-1, 1], the data scale is unified, and then the trend features are extracted through a one-dimensional convolutional neural network and input into a long short-term memory network, and jointly modeled with the time series features of the vibration signal to output a 256-dimensional joint time series feature vector of the vibration, temperature, and pressure signals. , comprehensively characterize the timing characteristics of device operation.

[0020] Preferably, when preprocessing the text data, the structured fields are classified based on the pre-trained BERT model, and the key entities are accurately identified, including: standard equipment components, fault types, and parameter indicators; then the RoBERTa-base model is used to extract the relationship between the entities, generate the "component-relationship-indicator" triples, including: "bearing-wear-vibration amplitude", and then output the 1024-dimensional text semantic feature vector , deeply explore the fault semantic information behind the text.

[0021] Preferably, when preprocessing the image data, the infrared thermal image is normalized to 512×512 pixels, and the pixel values are normalized to [-1, 1] to enhance the consistency of image features; then the spatial features of the infrared thermal image are extracted through the ResNet-50 convolutional neural network, and a 2048-dimensional image spatial feature vector is output. , clearly presenting the spatial characteristics of the surface temperature distribution of the equipment.

[0022] Preferably, the cross-modal alignment network through the dual-stream Transformer architecture described in step S2 specifically includes: using the dual-stream Transformer architecture to achieve spatiotemporal feature alignment, and dynamically fusing joint temporal features, text semantic features, and image spatial features through a gated attention mechanism.

[0023] Preferably, the dual-stream Transformer architecture includes two parallel branches, including a temporal stream branch and a spatial semantic stream branch.

[0024] Preferably, the time stream branch processes sensor data with time series characteristics, and inputs the joint time series feature vector To the 3-layer Transformer encoder stack structure, each layer contains a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism can capture the correlation between time series features from different angles. In order to capture the time series dependency, a sinusoidal time series position encoding is introduced. Through the formula and Generate position embeddings, where , and add it element by element with the input feature, and finally output the temporal feature enhancement vector , depicting the dynamic change trend of equipment operation.

[0025] Preferably, the spatial semantic flow branch processes image data with spatial characteristics and text data with semantic characteristics, first converting the 1024-dimensional text semantic feature vector and 2048-dimensional image space feature vector The vectors are concatenated into 3072 dimensions and then compressed to 256 dimensions through a linear layer to achieve modality dimension alignment. The vectors are then passed through a two-layer Transformer encoder for cross-modal interaction. Each layer uses a cross-attention mechanism to make text and image features mutually aware of each other. At the same time, a learnable modality type encoding is introduced, including text embedding. and image embedding ,pass and Enhance modality specificity and ultimately output a spatial semantic feature enhancement vector ,fusing fault semantics with heatmap spatial information.

[0026] Preferably, the spatiotemporal feature alignment is performed by outputting a temporal feature enhancement vector in each layer of the encoder in a time stream through the cross-branch attention mechanism of the dual-stream Transformer. Output temporal feature enhancement vector with spatial semantic flow Spliced into 512-dimensional vector As input, the time stream output is used as query and the spatial semantic stream output is used as key-value pair to calculate the mutual attention score ,in =256 is the feature vector dimension, which is consistent with the time stream branch dimension. Through cross-modal interaction, cross-modal association of vibration time series features with infrared thermal image spatial features and maintenance log semantic features is achieved.

[0027] Preferably, the dynamic fusion stage is based on the 512-dimensional vector achieved by the spatiotemporal feature alignment completed in step S223 , using gated attention mechanism for dynamic fusion, through the trainable matrix and bias Perform linear transformation and then generate a 256-dimensional gated value vector through the Sigmoid function , the calculation formula is: , where each element of g corresponds to the weight of the sensor time series feature, The joint weights of the corresponding text and image features are finally calculated through the dynamic weighting formula , realize adaptive modal fusion based on real-time data characteristics, and generate joint embedding representation containing spatiotemporal correlation ,in The equipment health index calculation used in step S4 provides a characteristic basis for equipment health assessment.

[0028] Preferably, the method for constructing the dynamic causal graph in step S3 is to construct a dynamic causal graph including a data layer, a feature extraction layer, and a causal reasoning layer to analyze the equipment fault propagation path.

[0029] Preferably, the data layer stores the physical topology data of the device in a static adjacency matrix The form of sensor joint time series feature vector The missing values in the sensor data are stored in a time series database and are filled using cubic spline interpolation. This method can reasonably estimate missing values while ensuring data smoothness. Outliers are removed according to the 3σ principle to ensure data reliability. The time window is then divided, and the sensor data is divided into 1-hour sliding windows. Each window generates a sequence containing 72-hour sliding window features. ,in ,Preserving the temporal continuity of the device status provides historical data support for ,subsequent analysis.

[0030] Preferably, the feature extraction layer uses a 2-layer graph convolutional neural network to extract features from the data output by the data layer, and the calculation of each layer is based on the formula Where l is the number of layers and takes values of 0 and 1. is the set of neighboring nodes of device i, is the trainable weight matrix, The ReLU activation function is used to combine the 128-dimensional node features output by the graph convolutional neural network with the 1024-dimensional text semantic feature vector compressed to 128 dimensions by the linear layer. After concatenation, a 256-dimensional vector is obtained. The fused feature vector contains the device physical topology and text semantic information, which will serve as the input of the subsequent causal reasoning layer.

[0031] Preferably, the causal reasoning layer performs a conditional independence test on the sensor data and the physical topology data based on the PC algorithm, constructs an initial undirected graph, and then determines the direction of the edge to obtain a directed acyclic graph as the initial causal network; then injects 23 domain rules structured in the form of conditions-results, including: "When the lubricating oil temperature is > 85°C, the probability of bearing failure increases by 4 times". These rules are based on professional domain knowledge, and the "component-relationship-indicator" triples generated by text semantic features are matched with the entities in the rules, and the network structure and edge relationships are modified to reflect the rule constraints; the initial causal network is modeled as a Bayesian network, and the Bayesian network parameter learning algorithm is used to combine the domain rules and data to estimate the conditional probability distribution of the nodes, and the rules are converted into prior probability constraints of the causal edges; the edges in the initial causal network are then tested using the d-separation criterion, and pseudo causal paths are deleted to ensure the accuracy of the causal relationship; at the same time, the GraphSAGE algorithm is used to aggregate real-time features and historical characteristics , according to the formula Update the weights of the edges in the causal graph, where MLP is a 3-layer fully connected network. is the time decay factor, and are the real-time feature vectors of nodes i and j respectively. The final output is a dynamic causal graph containing dynamic edge weights, which provides a basis for fault tracing for maintenance decisions and clearly presents the causal relationship and propagation path of equipment failures.

[0032] Preferably, the specific implementation of the improved proximal policy optimization algorithm in step S4 includes: first defining a state space including equipment health status and operation constraints, secondly designing an action space covering multiple maintenance operations, and then constructing a reward function that takes into account reliability, repair efficiency and cost, and finally generating a maintenance strategy through time series feature processing and policy optimization.

[0033] Preferably, the state space defines a state vector , where the equipment health index is calculated using a logistic regression model, using the formula: ,in is the multimodal fusion vector The weighted sum of The SHAP value is dynamically adjusted daily to reflect the contribution of each modal feature to the equipment's health status. θ is a preset threshold, MTBF is the mean time between failures, and is used to measure equipment reliability. The spare parts inventory adequacy ratio is calculated as the number of available spare parts divided by the number of maintenance requests. This state vector encompasses the key factors influencing equipment maintenance decisions and provides an accurate description of equipment status for subsequent decision-making.

[0034] Preferably, the action space construction includes 12 standard maintenance actions, including lubrication, tightening, replacement of spare parts and waiting, each action is associated with a cost coefficient The larger the cost coefficient value, the higher the cost of the maintenance action. At the same time, these actions are also subject to some constraints, including the hard constraint of lubrication interval ≥ 72h to avoid equipment loss caused by excessive maintenance; before performing replacement actions, the spare parts inventory must be greater than or equal to the safety threshold to ensure that the maintenance operation can be performed and prevent maintenance plan delays due to insufficient spare parts.

[0035] Preferably, the reward function design adopts a three-dimensional weighted reward function to guide strategy optimization, balancing equipment reliability, maintenance efficiency and cost control, and the formula is: , where MTTR is the mean time to repair, 、 and is the historical benchmark value. 、 and is the real-time value after executing the action, is an indicator function. If the failure probability output by the dynamic causal graph is less than 0.5 but a failure actually occurs, the function takes the value of 1, otherwise it takes the value of 0, strengthening the strategic penalty for missed faults.

[0036] Preferably, the reliability dimension controls rewards by increasing the mean time between failures ratio, encouraging the extension of equipment trouble-free operation time; the maintenance efficiency dimension controls rewards by attenuating the mean repair time ratio, guiding the reduction of fault repair time; the cost dimension controls penalties for high-consumption actions by the maintenance cost ratio; and the fault warning penalty item strengthens the strategic penalty for missed faults, prompting the model to predict faults more accurately and improving the scientific nature of the maintenance strategy.

[0037] Preferably, the strategy optimizes the state sequence of the previous 72 hours by a long short-term memory network. Perform temporal feature extraction to obtain hidden state ; Then the proximal strategy optimization algorithm is used based on the current state and hidden state , through the policy network Generate an action probability distribution; when generating an action probability distribution, the maintenance action corresponding to the root cause node of the dynamic causal graph will be prioritized. For example, when the root cause node is "bearing", maintenance actions related to bearings will be prioritized, such as bearing lubrication and maintenance, to enhance the relevance of the strategy and fault tracing; finally, the decision path is simulated using Monte Carlo tree search, and the advantage function is calculated through generalized advantage estimation. , taking into account the current state value and future reward discount, the formula is , where γ is the discount factor, The advantage function is the key input for policy updating. Combined with the clipping parameter 0.2, it limits the policy update amplitude to avoid violent policy fluctuations during training and ensure the stable convergence of the optimization process, thereby generating high-quality maintenance strategies, including routine inspection strategies, preventive maintenance strategies, and emergency shutdown strategies, and realizing intelligent and adaptive maintenance decision-making.

[0038] Preferably, the triggering logic of the three-level warning mechanism in step S5 is as follows: when the equipment health index is ≥0.7, it is in the first-level warning, based on the "routine inspection" strategy, an inspection instruction is issued to the DCS system through the OPC UA protocol, and an equipment health report containing the failure probability and health trend is generated. It is recommended to complete the manual inspection within 3 days. The instruction content is determined by the high-risk nodes output by the dynamic cause-and-effect diagram, including the inspection route and key monitoring components, so that the inspection work is more targeted and efficient; when 0.4≤equipment health index<0.7, the second-level warning is triggered, the "preventive maintenance" branch in the maintenance strategy is triggered, and the spare parts inventory system is automatically checked. If the inventory adequacy rate is <80%, a spare parts purchase work order is generated, and a preventive maintenance work order containing the maintenance type and execution time is generated at the same time. The equipment downtime window is locked through the OPC UA protocol to prevent potential failures in advance and reduce the impact of equipment failures on production; when the equipment health index is <0.4, it enters the third-level warning, activates the "emergency shutdown" plan in the maintenance strategy, and activates the "emergency shutdown" plan through the OPC The UA protocol sends the highest-priority hard shutdown command to the DCS system, and simultaneously pushes the root cause node information from the dynamic cause-and-effect graph analysis to the maintenance terminal, initiating the emergency spare parts allocation process to minimize the losses caused by equipment failure.

[0039] Based on the above method, the present invention also constructs a corresponding power plant intelligent maintenance system based on multimodal dynamic graph learning, which includes: a data acquisition layer, an edge computing layer, a cloud-based intelligent analysis layer, and a decision execution layer.

[0040] Preferably, the data acquisition layer is responsible for deploying multi-source sensor clusters and industrial terminals, executing step S1 to collect structured sensor data, unstructured text data, image data and device physical topology data of device operation.

[0041] Preferably, the edge computing layer deploys computing nodes and executes step S21 to pre-process the collected data, including pre-processing of vibration signals, temperature signals, pressure signals, text data and image data, and caches the feature vectors of the last 72 hours, and transmits the pre-processed feature vectors to the cloud through the OPC UA protocol. 、 、 , reduce cloud computing pressure and improve data processing efficiency.

[0042] Preferably, the cloud-based intelligent analysis layer integrates a multimodal fusion engine, a dynamic graph training cluster, and a reinforcement learning decision module, wherein the multimodal fusion engine executes step S22 based on a dual-stream Transformer architecture and outputs a joint embedding representation containing spatiotemporal correlations. To the reinforcement learning decision module; the dynamic causal graph training cluster executes step S3, outputs status information including equipment failure probability, root cause node set, and causal relationship strength to the reinforcement learning decision module; the reinforcement learning decision module executes step S4, based on the output of the multimodal fusion engine The equipment status information output by the dynamic causal graph training cluster is used to generate maintenance strategies through the improved PPO algorithm, and the maintenance strategies are output to the decision execution layer.

[0043] Preferably, the decision execution layer receives the maintenance strategy output by the reinforcement learning decision module through the OPC UA protocol, triggers the third level warning according to the strategy type, and at the same time, the decision execution layer interacts with the DCS system to execute the maintenance instruction of step S5.

[0044] Preferably, the decision execution layer is equipped with a maintenance knowledge base and a visual human-computer interaction interface. The maintenance knowledge base stores information related to equipment maintenance, including historical data, fault cases, and maintenance manuals, which can be used for operator reference to help them quickly understand the past status of the equipment and corresponding treatment measures; the visual human-computer interaction interface has an intuitive graphical display function, supports the issuance of maintenance instructions, can clearly display the root cause analysis results of the fault, provide decision support for maintenance personnel, and improve the efficiency and accuracy of maintenance work.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. This invention utilizes a cross-modal attention mechanism and a two-stream Transformer architecture to achieve efficient fusion of multimodal data. For sensor data processing, a one-dimensional convolutional neural network is first used to extract local features of vibration signals. A bidirectional long short-term memory network is then used to capture temporal dependencies, accurately capturing dynamic changes in equipment operation. A RoBERTa-base model is used to extract semantic meaning from text data, mining fault clues from maintenance logs. A ResNet-50 is used to extract spatial features from image data, capturing the surface temperature distribution of the equipment. Finally, a gated attention mechanism dynamically assigns weights to each modal feature based on real-time data characteristics, enabling adaptive fusion and generating a joint embedding representation with only 256 dimensions. This fusion approach successfully integrates structured and unstructured data, avoiding the curse of dimensionality caused by traditional methods that directly concatenate data. Compared to some earlier fusion methods, which experience a decline in model accuracy when the measured feature dimension exceeds 5000, this invention significantly improves feature fusion efficiency, providing more accurate feature information for subsequent equipment health index calculations and maintenance decisions.

[0047] 2. The present invention constructs a dynamic causal graph, generates a static adjacency matrix based on the physical connection of the equipment, and uses the GraphSAGE algorithm combined with real-time data and time decay factors to update edge weights in real time. This can effectively capture the dynamic associations between devices and adapt to changes in equipment operating conditions. At the same time, a causal network is constructed based on the PC algorithm, and 23 domain knowledge rule bases are injected. The "component-relationship-indicator" triples generated by combining text semantic features are matched with rule entities, and the d-separation criterion is used to eliminate false causal relationships and improve the accuracy of fault root cause identification. Compared with traditional equipment relationship modeling methods, it can locate the source of the fault more quickly and accurately, provide support for preventive maintenance, and reduce the impact of equipment failure on production.

[0048] 3. The present invention introduces an adaptive reinforcement learning decision engine, which comprehensively reflects the equipment status by defining a state vector that includes multiple factors such as equipment health index and load rate. The designed three-dimensional reward function comprehensively considers the three key dimensions of mean time between failures, mean time to repair, and cost. Through reasonable weight distribution, it guides the generation of a more scientific maintenance strategy, effectively balancing equipment reliability, maintenance efficiency, and cost control. The proximal policy optimization algorithm combines long and short-term memory neural networks to process historical state sequences, which can fully utilize the historical operation information of the equipment and integrate Monte Carlo tree search to simulate decision scenarios, providing a more comprehensive reference for policy optimization. At the same time, a clipping parameter of 0.2 is used to limit the update amplitude during the policy update process to avoid violent fluctuations in the strategy during training and ensure stable convergence of the optimization process. This enables maintenance decisions to adapt to equipment changes in real time, effectively reduces the false alarm rate, and improves the scientific and dynamic nature of decision-making. Compared with traditional fixed thresholds or rule-based decision-making systems, it can better cope with the complex and changeable operating environment of power plant equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a system architecture diagram of the present invention;

[0050] Figure 2 It is a dynamic graph construction flow chart of the present invention;

[0051] Figure 3 It is a reinforcement learning decision sequence diagram of the present invention;

[0052] Figure 4 It is a schematic diagram of the specific application of the nuclear safety rules of the present invention in the dynamic cause-effect diagram;

[0053] Figure 5 Schematic diagram of the multimodal data fusion process of the present invention;

[0054] Figure 6 This is an example diagram of the equipment fault propagation path of the present invention. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0057] Example 1

[0058] refer to Figure 1 、 Figure 2 、 Figure 3 and Figure 5 The implementation steps of the power plant intelligent maintenance method and system based on multimodal dynamic graph learning of the present invention include: multi-source data acquisition, multimodal data fusion, dynamic causal graph construction, reinforcement learning decision-making, and maintenance plan generation and execution.

[0059] During the multi-source data collection phase, in order to comprehensively obtain equipment operating status information, the data collection layer deployed multi-source sensor clusters and industrial terminals at key locations of power plant equipment to collect real-time structured sensor data, unstructured text data, and image data of equipment operation. It also constructed static connection relationships between devices based on power plant design drawings and historical records as equipment physical topology data.

[0060] Specifically, the multi-source sensor cluster includes vibration sensors, temperature sensors, and pressure sensors. The sensor data includes vibration signals, temperature signals, and pressure signals. The vibration sensors use ICP accelerometers to collect vibration signals. There are 18 of them in total. They are distributed in key areas such as turbines and boilers according to the equipment's vibration-sensitive areas and heat concentration areas. They can comprehensively capture vibration changes in different parts of the equipment during operation. The sensor sampling frequency on the main equipment is set to 1kHz, and that on the auxiliary equipment is 200Hz. The frequency response range is 0.5Hz-10kHz, which is used to capture early vibration signs of potential equipment failure. The temperature sensors use high-precision fiber optic sensors. One is deployed every 5 meters along the steam pipeline. At key areas prone to abnormal temperature changes, such as elbows and valves, the number of sensors is increased to one every 2 meters. A total of 56 high-precision fiber optic sensors are deployed. The spatial resolution of these high-precision fiber optic sensors reaches 1m, and the temperature measurement accuracy is ±0.5°C. The pressure sensors use piezoresistive pressure sensors. They are deployed at 12 key pressure monitoring points on the boiler drum. The range of the piezoresistive pressure sensors is 0-25MPa, and the accuracy reaches 0.1%. FS, collects boiler drum pressure data in real time and outputs it to the programmable logic controller in the form of a 4-20mA signal.

[0061] Specifically, industrial terminals include industrial explosion-proof tablets and infrared cameras. Text data is maintenance logs entered through industrial explosion-proof tablets. Speech-to-text technology is used to convert maintenance records into structured fields, and specific regular expression templates are used, including "equipment: [bearing|gearbox], fault: [vibration|temperature rise], level: [I-IV]", to quickly extract key information; image data is infrared thermal images collected by infrared cameras. The infrared cameras are installed near the key heat dissipation areas and parts prone to failure of the equipment. The resolution is 640×480 and the thermal sensitivity is ≤0.03℃. Infrared thermal images are collected at a frequency of once every 30 minutes. Once the equipment shows signs of abnormal heating, the system automatically switches to collecting data once a minute, and accurately determines whether there are abnormal heating areas in the equipment based on temperature differences.

[0062] In the multimodal data fusion stage, the collected multimodal data is input into the edge computing layer for data preprocessing. Then, through the dual-stream Transformer architecture cross-modal alignment network, the spatiotemporal features of sensor data, text data, and image data are aligned and fused to generate a joint embedding representation containing spatiotemporal associations. .

[0063] Specifically, at the edge computing layer, three high-performance edge computing devices are used to build a redundant cluster. The nodes monitor each other through a heartbeat detection mechanism, and the detection interval is set to 5 seconds. Each node is equipped with a high-speed data transmission interface and a large-capacity cache to ensure the stability of data transmission and the timeliness of data processing.

[0064] Specifically, when the master node is working normally, it is responsible for receiving and processing various types of data from the data acquisition layer; once the master node fails, the backup node quickly completes the switch within 10 seconds and takes over the data processing work.

[0065] Specifically, the switching logic is that each node sends a heartbeat signal to other nodes every 5 seconds to indicate that it is in normal operation. Once a node does not receive the heartbeat signal from the master node within the specified 5 seconds, the system will determine that the master node may have a fault. After determining that the master node has failed, the backup node begins to prepare to take over and preload the necessary data and program resources. The preloaded resources include various key data and programs required for system operation. Within 10 seconds after determining that the master node has failed, the backup node completes the switching work and takes over the data processing work from the master node to ensure the continuity of data processing.

[0066] Specifically, during the data preprocessing process, the vibration signal in the sensor data is first subjected to a 1024-point fast Fourier transform to convert the time domain signal into frequency domain features and highlight the signal frequency components. Then, local features are extracted through a one-dimensional convolutional neural network. The convolution kernel is optimized according to the vibration characteristics of the equipment to capture the key changes in the vibration signal. After that, the timing dependencies are captured through a bidirectional long short-term memory network.

[0067] Specifically, the temperature and pressure signals in the sensor are normalized, and the value range is mapped to [-1, 1] to unify the data scale. Then, the trend features are extracted through a one-dimensional convolutional neural network and input into a long short-term memory network. The model is jointly built with the time series features of the vibration signal to output a 256-dimensional joint time series feature vector. .

[0068] Specifically, the text data is based on the BERT model pre-trained on 5,000 annotated work orders in the power plant to perform entity classification on structured fields, identifying key entities including standard equipment components, fault types, and parameter indicators. The RoBERTa-base model is then used to extract relationships between entities, generate "component-relationship-indicator" triplets, and then output a 1024-dimensional text semantic feature vector. .

[0069] Specifically, the image data normalizes the infrared thermal image to 512×512 pixels, and normalizes the pixel values to [-1, 1] to enhance the consistency of image features. Then, the spatial features of the infrared thermal image are extracted through the ResNet-50 convolutional neural network, and a 2048-dimensional image spatial feature vector is output. .

[0070] The multimodal data after data preprocessing is transmitted to the cloud-based intelligent analysis layer. In the cloud-based intelligent analysis layer, cross-modal alignment and fusion are performed through the multimodal fusion engine based on the dual-stream Transformer architecture, and a joint embedding representation containing spatiotemporal correlations is output to the reinforcement learning decision module; the dynamic causal graph training cluster executes the dynamic causal graph construction and outputs status information to the reinforcement learning decision module; the reinforcement learning decision module generates a maintenance strategy based on the equipment status information output by the multimodal fusion engine and the dynamic causal graph training cluster through the improved PPO algorithm, and outputs the maintenance strategy to the decision execution layer.

[0071] Cross-modal alignment and fusion uses a two-stream Transformer architecture to achieve spatiotemporal feature alignment, and dynamically fuses temporal features, text semantic features, and image spatial features through a gated attention mechanism.

[0072] Specifically, the two-stream Transformer architecture consists of a temporal stream branch and a spatial semantic stream branch.

[0073] Specifically, the time stream branch processes sensor data with time series characteristics and inputs the joint time series feature vector To the 3-layer Transformer encoder stack structure, each layer contains a multi-head self-attention mechanism and a feedforward neural network. In order to capture the temporal dependency, a sinusoidal temporal position encoding is introduced. and Generate position embeddings, where , add it to the input feature element by element, and finally output the temporal feature enhancement vector .

[0074] Specifically, the spatial semantic flow branch processes image data with spatial characteristics and text data with semantic characteristics. First, the 1024-dimensional text semantic feature vector and 2048-dimensional image space feature vector The concatenation is done into a 3072-dimensional vector, which is compressed to 256 dimensions through a linear layer to achieve modality dimension alignment. Then, a two-layer Transformer encoder is used for cross-modal interaction. Each layer uses a cross-attention mechanism to make text and image features mutually aware of each other, and introduces learnable modality type encoding, including text embedding. and image embedding ,pass and Enhance modality specificity and ultimately output a spatial semantic feature enhancement vector ,fusing fault semantics with heatmap spatial information.

[0075] Specifically, in the spatiotemporal feature alignment stage, the cross-branch attention mechanism of the two-stream Transformer is used to output the temporal feature enhancement vector in each layer of the encoder in a time stream. Output feature enhancement vector with spatial semantic flow Spliced into 512-dimensional vector As input, the time stream output is used as query and the spatial semantic stream output is used as key-value pair to calculate the mutual attention score ,in =256 is the feature vector dimension, which is consistent with the time stream branch dimension. Through cross-modal interaction, cross-modal association of vibration time series features with infrared thermal image spatial features and maintenance log semantic features is achieved.

[0076] Specifically, the dynamic fusion stage is based on the 512-dimensional vector achieved by the above-mentioned spatiotemporal feature alignment. , using gated attention mechanism for dynamic fusion, through the trainable matrix and bias Perform linear transformation and then generate a 256-dimensional gated value vector through the Sigmoid function , the calculation formula is: , where each element of g corresponds to the weight of the sensor time series feature, The joint weight of the corresponding text and image features; finally, the dynamic weighted formula , realize adaptive modal fusion based on real-time data characteristics, and generate joint embedding representation containing spatiotemporal correlation ,in Used for subsequent equipment health index calculation.

[0077] Dynamic causal graph construction based on the joint time series feature vector of device physical topology data and sensor data , combined with text semantic feature vector Perform domain rule matching and build a dynamic causal graph consisting of a data layer, a feature extraction layer, and a causal reasoning layer. Identify and output the device status information, which includes the current failure probability of each device, the failure propagation path, the root cause node information, and the strength of the causal relationship between devices.

[0078] Specifically, the data layer stores the device physical topology data in a static adjacency matrix. The form of sensor joint time series feature vector The sensor data is stored in a time series database. Missing values in the sensor data are filled using cubic spline interpolation. Outliers are removed based on the 3σ principle. The time window is then divided, and the sensor data is split into 1-hour sliding windows. Each window generates a sequence containing 72-hour sliding window features. ,in .

[0079] Specifically, the feature extraction layer uses a two-layer graph convolutional neural network to extract features from the data output by the data layer. The calculation of each layer is based on the formula Here l is the number of layers, l takes values of 0 and 1, is the set of neighboring nodes of device i, is the trainable weight matrix, The ReLU activation function is used to combine the 128-dimensional node features output by the graph convolutional neural network with the 1024-dimensional text semantic feature vector compressed to 128 dimensions by the linear layer. After concatenation, a 256-dimensional vector is obtained. The fused feature vector contains the device physical topology and text semantic information, which serves as the input of the subsequent causal reasoning layer.

[0080] Specifically, the causal reasoning layer performs conditional independence tests on sensor data and physical topology data based on the PC algorithm, constructs an initial undirected graph, and then determines the direction of the edges to obtain a directed acyclic graph as the initial causal network. It then injects 23 domain rules structured in the form of conditions and results, modifies the network structure and the relationship between the edges to reflect the rule constraints, and models the initial causal network as a Bayesian network. The Bayesian network parameter learning algorithm is used to combine domain rules and data to estimate the conditional probability distribution of nodes, and the rules are converted into prior probability constraints on causal edges. The edges in the initial causal network are then tested using the d-separation criterion, pseudo causal paths are deleted, and the GraphSAGE algorithm is used to aggregate real-time features. and historical characteristics , according to the formula Update the weights of the edges in the causal graph, where MLP is a 3-layer fully connected network. is the time decay factor, and are the real-time feature vectors of nodes i and j respectively, and the final output is a dynamic causal graph containing dynamic edge weights.

[0081] Specifically, the 23 domain rules are based on professional domain knowledge. The "component-relationship-indicator" triples generated by text semantic features are matched with the entities in the rules, and the network structure and edge relationships are modified to reflect the rule constraints.

[0082] Specifically, when the "vibration amplitude" of the "bearing" component is detected to be abnormal and the "lubricating oil temperature > 85°C" condition is triggered, the system automatically activates the corresponding rule, adjusts the weight of the "bearing→fault" edge in the causal network, and enhances the causal strength of the path.

[0083] Reinforcement learning decision-making is based on the joint embedding representation generated by the device status information output by the dynamic causal graph and the fusion of multimodal data , to calculate the equipment health index, and through the improved proximal policy optimization algorithm, the state vector containing the dynamic causal graph state and the three-dimensional reward function are used as input to generate the maintenance strategy to achieve multi-objective optimization.

[0084] Specifically, the improved proximal policy optimization algorithm first defines a state space that includes equipment health status and operating constraints. Secondly, it designs an action space covering various maintenance operations. Then, it constructs a reward function that takes into account reliability, repair efficiency, and cost. Finally, it generates a maintenance strategy through time series feature processing and policy optimization.

[0085] Specifically, the state space defines the state vector , where the equipment health index is calculated using a logistic regression model, using the formula: ,in, is represented from the joint embedding The features related to device health extracted from is the corresponding weight, and θ is the preset threshold.

[0086] Specifically, the action space .

[0087] Specifically, the reward function Taking into account factors such as equipment safety, reliability, and maintenance costs, the formula is: , where S is the safety score, which is calculated based on the equipment failure probability and fault propagation path in the dynamic causal diagram; R is the reliability score, which is related to the mean time between failures (MTBF) of the equipment; C is the cost score, which takes into account maintenance costs and spare parts costs; MTTR is the mean time to repair score; and E is the environmental adaptability score, which takes into account factors such as ambient humidity and coal quality fluctuations.

[0088] Specifically, during the training process, an adaptive learning rate is used for strategy optimization. The initial learning rate is 0.001. When the average reward fluctuation for 10 consecutive rounds is less than 0.01, the learning rate is reduced to 0.8 times the original rate. Through multiple iterative training, the reinforcement learning decision module continuously optimizes the strategy to maximize the long-term cumulative reward.

[0089] After receiving the maintenance strategy output by the reinforcement learning decision module, the decision execution layer interacts with the power plant's distributed control system (DCS), equipment management system (EMS), etc. through the enterprise service bus (ESB).

[0090] Specifically, for maintenance strategies including inspections, minor repairs, medium repairs, and major repairs, the system automatically generates detailed work orders, which include maintenance task descriptions, required spare parts lists, maintenance personnel arrangements, estimated maintenance time, etc.

[0091] Specifically, maintenance personnel receive work orders through mobile terminal devices and use augmented reality (AR) technology for on-site maintenance guidance. AR devices can display the three-dimensional model of the equipment, fault location, repair steps and other information to help maintenance personnel complete maintenance tasks quickly and accurately.

[0092] Specifically, the decision-making execution layer is equipped with an equipment maintenance knowledge base and a visual human-computer interaction interface. The equipment maintenance knowledge base stores a large amount of information, including equipment failure cases, maintenance experience, and technical documents. It uses natural language processing technology to realize intelligent search and recommendation functions, making it convenient for maintenance personnel to quickly obtain the required information; the visual human-computer interaction interface uses 3D visualization technology to display the operating status, maintenance plan, fault warning and other information of power plant equipment, and supports multi-dimensional data analysis and decision-making assistance functions.

[0093] Example 2

[0094] refer to Figures 1 to 4 In view of the special operating environment and high reliability requirements of nuclear power equipment, the present invention has made targeted adjustments in this embodiment.

[0095] Specifically, at the data acquisition layer, in addition to the vibration, temperature, and pressure sensors deployed in conventional power plants, radiation monitoring sensors are added and deployed around key equipment including reactor pressure vessels, steam generators, and main pumps. According to the distribution characteristics of the radiation field, 15 radiation monitoring sensors are evenly distributed to monitor the changes in radiation dose around the equipment in real time. The measurement range of the radiation monitoring sensors is , the measurement accuracy is To ensure data collection stability and security in high-radiation environments, vibration and temperature sensors are designed with high radiation resistance, capable of withstanding radiation dose rates up to 10^4 Gy / h. Maintenance work order text entry utilizes an industrial explosion-proof flat panel with an IP68 rating, effectively preventing radiation, dust, and moisture intrusion.

[0096] Specifically, in order to effectively integrate radiation data with other types of data for analysis, a normalization method is used. Perform data preprocessing on radiation data.

[0097] Specifically, to ensure the stability and security of data collection in high radiation environments, vibration and temperature sensors are selected with strong radiation resistance, and their radiation dose resistance rate can reach ,The maintenance work order text entry uses an industrial explosion-proof flat panel with a higher protection level of IP68.

[0098] Specifically, the collected data is transmitted to the edge computing layer. The edge computing layer of this embodiment also uses high-performance edge computing equipment, but optimizes and upgrades the data processing algorithm.

[0099] Specifically, not only conventional data preprocessing is performed on vibration, temperature and other signals, but also real-time analysis and filtering of radiation monitoring data is required to remove noise interference and extract effective information.

[0100] Specifically, for radiation monitoring data, the wavelet transform method is used for denoising. The wavelet basis function is db4, and the number of decomposition layers is 3. When performing feature extraction, in addition to extracting conventional time domain and frequency domain features, features including the radiation dose change rate and cumulative radiation dose are also extracted for the radiation data. The processed data is then transmitted to the cloud-based intelligent analysis layer via a high-speed fiber optic network, with a transmission rate of up to 10 Gbps.

[0101] Specifically, at the cloud-based intelligent analysis layer, the multimodal fusion engine enhances the fusion effect of radiation data and other equipment operation data by adjusting the parameters of the cross-modal alignment network, enabling the system to better explore the hidden potential connections between radiation data and equipment failures.

[0102] Specifically, in terms of constructing dynamic causal graphs, we fully combine the special process flow and strict safety regulations of nuclear power equipment, and add 12 new nuclear safety constraint rules on the basis of the original domain knowledge rules, including "coolant pressure <15MPa and core temperature >350℃ → forced shutdown".

[0103] Specifically, nuclear safety constraint rules are integrated into causal reasoning through the following process: first, threshold detection is performed on real-time sensor data, including coolant pressure and core temperature. When "pressure < 15MPa" and "temperature > 350℃" are simultaneously met, the rule matching module is triggered, and a forced causal edge of "safety system → forced shutdown" is automatically generated in the causal graph. The weight of this edge is set to the maximum value of 1.0, and other non-safety-related causal paths are blocked to ensure the priority execution of nuclear safety rules.

[0104] Specifically, when constructing the causal network, we still follow the PC algorithm, injection rules, modeling Bayesian networks, and using the d-separation criterion to ensure the accuracy of causal relationships.

[0105] Specifically, the reinforcement learning decision engine added radiation dose-related indicators to the state space definition, including the cumulative radiation dose with a weight set to 0.1 and the radiation dose change rate with a weight set to 0.15, allowing maintenance decisions to more comprehensively consider nuclear radiation-related factors.

[0106] Specifically, to highlight the importance of safety factors, the reward function increases the safety weight to 0.4, and the original MTBF, MTTR and cost weights are adjusted to 0.4, 0.2 and 0.2 respectively, to ensure that nuclear safety is always given priority in the maintenance decision-making process.

[0107] Specifically, the three-level early warning mechanism is divided into detailed levels according to the safety level of nuclear power equipment.

[0108] Specifically, the first level warning is a low-risk warning, which is triggered when certain non-critical parameters of the equipment show minor anomalies, such as a small fluctuation in radiation dose within the normal range (a rate of change of less than 5%) or a temperature slightly above the normal range (no more than 5°C). At this time, the system will automatically send an early warning message to the mobile phones and computer terminals of relevant operation and maintenance personnel, reminding them to pay attention to the equipment status and conduct an inspection of the equipment within 24 hours. The second level warning is a medium-risk warning, which is triggered when key parameters of the equipment show obvious anomalies, such as a radiation dose change rate exceeding 5% but less than 10% or a coolant pressure drop of 5%-10%. The system will immediately activate the emergency plan, automatically adjust the equipment's operating parameters, and notify professional technicians to arrive at the site within 4 hours for a detailed inspection and assessment. The third level warning is a high-risk warning, which is triggered when a situation seriously affecting nuclear safety occurs, including a sharp increase in radiation dose (a rate of change exceeding 10%), a core temperature exceeding 350°C, or a coolant pressure below 15 MPa. At this time, the system will immediately initiate the emergency shutdown procedure, swiftly organize professional personnel to conduct a comprehensive troubleshooting and repair, and report to the higher-level authorities and relevant regulatory agencies.

[0109] Specifically, the maintenance knowledge base and visual human-computer interaction interface equipped at the decision-making and execution layer are optimized for the characteristics of nuclear power equipment. In addition to storing conventional equipment failure cases and maintenance experience, the maintenance knowledge base also adds a large amount of nuclear safety-related knowledge and emergency plans, including nuclear radiation protection knowledge and nuclear accident handling procedures. It uses semantic search technology to support users to query knowledge through natural language, improving the efficiency of knowledge acquisition. The visual human-computer interaction interface uses virtual reality (VR) and augmented reality (AR) technologies to provide operation and maintenance personnel with a more intuitive and immersive equipment monitoring and maintenance experience. Operation and maintenance personnel can remotely view the internal structure and operating status of the equipment through VR equipment and perform equipment maintenance and troubleshooting on-site through AR equipment. At the same time, the interface displays key parameters such as radiation dose, temperature, pressure, and early warning information of the equipment in real time, ensuring that operation and maintenance personnel can promptly grasp the safety status of the equipment.

[0110] Example 3

[0111] 1 , 2 , 3 , 5 and 6 , this embodiment adopts a combination of centralized and distributed approaches in a power generation cluster scenario including multiple thermal power equipment.

[0112] Specifically, the data acquisition layer has installed various sensors in accordance with standard configuration on each thermal power equipment to achieve independent monitoring of each device. Four vibration sensors and two temperature sensors are installed on the bearings of the steam turbine to monitor the vibration and temperature changes of the bearings; pressure sensors, temperature sensors and flow sensors are installed on the furnace, superheater, economizer and other parts of the boiler to monitor the operating parameters of the boiler in real time.

[0113] Specifically, in order to facilitate the subsequent unified processing and comprehensive analysis of data from multiple devices in the cluster, the feature data across devices is aligned based on the device code and timestamp to ensure that the data from different devices are consistent in time and device identification.

[0114] Specifically, the collected multi-source data is first transmitted to the edge computing layer. The edge computing layer deploys small edge computing devices near each device. These devices are responsible for preliminary preprocessing and feature extraction of local data. For vibration signals, the small edge computing devices use fast Fourier transform to convert time domain signals into frequency domain signals with 1024 sampling points. Then, the energy distribution of the frequency domain signal is calculated, and features such as peak frequency and center frequency are extracted. At the same time, the sliding average filtering method is used to smooth the signal and remove noise interference. The sliding window size is 10 sampling points. For temperature and pressure signals, the normalization method is used to map the data to the [0, 1] interval. The formula is: , where x is the original data, and They are the minimum and maximum values of the signal respectively. When performing feature extraction, statistical features such as the mean, variance, and standard deviation of the signal are extracted.

[0115] Specifically, in addition to using normalization methods to pre-process temperature signals, small edge devices also add periodic component extraction based on Fourier transform to target the periodic fluctuation characteristics of temperature signals, identify periodic trends such as 24 hours and 7 days, and combine them with real-time monitoring values to form a composite feature vector, thereby improving the robustness to seasonal load changes or ambient temperature influences and ensuring the characteristic stability under variable operating conditions such as unit start-up and shutdown.

[0116] Specifically, the data after preliminary processing is then transmitted to the centralized edge computing node in the cloud for further integration and analysis, which reduces the pressure of data transmission and improves the overall data processing efficiency.

[0117] Specifically, the cloud-based intelligent analysis layer builds a unified multimodal fusion model and dynamic causal graph to conduct a comprehensive analysis of the data of the entire thermal power equipment cluster. Through shared models and networks, it explores the potential correlations and fault propagation patterns between different devices.

[0118] Specifically, the reinforcement learning decision engine formulates a globally optimal maintenance strategy based on the equipment status and maintenance resource allocation of the entire cluster. For example, when multiple devices simultaneously face varying degrees of failure risk, the decision engine will comprehensively weigh factors such as the importance of the equipment, maintenance costs, and maintenance time, reasonably allocate maintenance resources, and give priority to equipment failures that have a greater impact on power generation production.

[0119] Specifically, in the decision-making process, the Hungarian algorithm is used to solve the multi-equipment maintenance task allocation problem, and the objective function is , while satisfying the constraints and ,in, is the cost of allocating the i-th equipment to the j-th maintenance team, is a binary variable that indicates whether the i-th equipment is assigned to the j-th maintenance team to achieve the optimal allocation of maintenance resources.

[0120] Specifically, the decision-making execution layer accurately sends maintenance instructions to the execution terminals of each device through a unified maintenance scheduling system.

[0121] Specifically, the decision-making execution layer has also established an equipment maintenance knowledge base and experience sharing platform. The storage format adopts the four-tuple <equipment type, failure mode, treatment measures, effect evaluation>. The knowledge base supports SQL query and case retrieval functions, which allows technicians to quickly obtain historical maintenance experience and provide a reference basis for maintenance decisions of current equipment, including: when a certain equipment has a specific failure mode, technicians can quickly query the treatment measures and effect evaluation of similar failures through the knowledge base, so as to formulate a more reasonable maintenance plan; the visual human-computer interaction interface displays the maintenance status of each equipment in real time, which makes it convenient for operators to intuitively understand the equipment status and supports the issuance of maintenance instructions. The interface adopts a responsive design and supports display on screens of different sizes, including computers, tablets and mobile phones. At the same time, the interface also provides data analysis and statistical functions, including equipment failure rate statistics and maintenance cost analysis, to help managers better understand the equipment maintenance status of the power generation cluster.

[0122] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A power plant intelligent maintenance method based on multimodal dynamic graph learning, characterized in that: The following steps are involved: S1. Multi-source data acquisition: Through multi-source sensor clusters and industrial terminals, real-time acquisition of device operation structured sensor data, unstructured text data, and image data; S2. Multimodal data fusion: First, preprocess the multimodal data to generate a joint time series feature vector of the sensor data. , text semantic feature vector and image space feature vector Then, through the dual-stream Transformer architecture cross-modal alignment network, the sensor data, text data and image data are aligned and fused with spatiotemporal features to generate a joint embedding representation containing spatiotemporal associations. ; S3. Dynamic Causal Graph Construction: Based on the Joint Time Series Feature Vector of Device Physical Topology Data and Sensor Data , combined with text semantic feature vector Perform domain rule matching to build a dynamic causal graph consisting of a data layer, a feature extraction layer, and a causal reasoning layer. This identifies and outputs device status information, including the current failure probability of each device, the failure propagation path, the root cause node information, and the strength of the causal relationship between devices. S4. Reinforcement Learning Decision-Making: Joint Embedding Representation Generated by Fusion of Device Status Information and Multimodal Data Output from Dynamic Causal Graph , to calculate the equipment health index, and through the improved proximal policy optimization algorithm, the state vector containing the dynamic causal graph state and the three-dimensional reward function are used as input to generate maintenance strategies to achieve multi-objective optimization. The maintenance strategy types include routine inspection strategy, preventive maintenance strategy and emergency shutdown strategy; S5. Maintenance plan generation and execution: Trigger the three-level warning mechanism based on the calculated equipment health index. Generate and execute corresponding specific instructions based on the specific maintenance strategy type generated in step S4. Different maintenance strategies correspond to specific actions in the action space.

2. The power plant intelligent maintenance method based on multimodal dynamic graph learning according to claim 1 is characterized in that: In step S1, the multi-source sensor cluster includes a vibration sensor, a temperature sensor, and a pressure sensor, and the sensor data includes a vibration signal, a temperature signal, and a pressure signal; the industrial terminal includes an industrial explosion-proof tablet and an infrared camera, and the text data is maintenance logs entered through the industrial explosion-proof tablet, and structured fields are generated using speech-to-text technology; the image data is an infrared thermal image collected by the infrared camera.

3. The power plant intelligent maintenance method based on multimodal dynamic graph learning according to claim 1 is characterized in that: The data collected in step S1 also includes equipment physical topology data, which is the static connection relationship between devices obtained based on power plant design drawings and historical records.

4. The power plant intelligent maintenance method based on multimodal dynamic graph learning according to claim 1 is characterized in that: The step S2 specifically includes: S21. Data Preprocessing: The multimodal data includes sensor data, text data, and image data. When preprocessing the sensor data, a 1024-point fast Fourier transform is performed on the vibration signal to convert the time domain signal into frequency domain features. Local features are then extracted using a one-dimensional convolutional neural network. Time series dependencies are captured using a bidirectional long short-term memory network. Temperature and pressure signals are normalized and mapped to a range of [-1, 1]. Trend features are extracted using a one-dimensional convolutional neural network and then input into a long short-term memory network. Joint modeling is performed with the time series features of the vibration signal to output a joint time series feature vector of the vibration, temperature, and pressure signals. When preprocessing the text data, the structured fields are classified into entities based on the pre-trained BERT model, including standard equipment components, fault types, and parameter indicators. The RoBERTa-base model is used to extract the relationship between the entities, generate "component-relationship-indicator" triples, and output a 1024-dimensional text semantic feature vector. When preprocessing the image data, the infrared thermal image is normalized to 512×512 pixels, and the pixel values are normalized to [-1, 1]. The spatial features of the infrared thermal image are extracted through the ResNet-50 convolutional neural network, and the image spatial feature vector is output. ; S22. Cross-modal alignment and fusion: A two-stream Transformer architecture is used to achieve spatiotemporal feature alignment. The gated attention mechanism is used to dynamically fuse temporal features, textual semantic features, and image spatial features. The two-stream Transformer architecture consists of two parallel branches, including a temporal stream branch and a spatial semantic stream branch.

5. The power plant intelligent maintenance method based on multimodal dynamic graph learning according to claim 4 is characterized in that: The step S22 specifically includes: S221. The time stream branch processes sensor data with time series characteristics and inputs the joint time series feature vector To the 3-layer Transformer encoder stack structure, each layer contains a multi-head self-attention mechanism and a feedforward neural network. In order to capture the temporal dependency, a sinusoidal temporal position encoding is introduced. and Generate position embeddings, where , add it to the input feature element by element, and finally output the temporal feature enhancement vector ; S222. The spatial semantic flow branch processes image data with spatial characteristics and text data with semantic characteristics. First, the 1024-dimensional text semantic feature vector and 2048-dimensional image space feature vector The concatenation is done into a 3072-dimensional vector, which is compressed to 256 dimensions through a linear layer to achieve modality dimension alignment. Then, a two-layer Transformer encoder is used for cross-modal interaction. Each layer uses a cross-attention mechanism to make text and image features mutually aware of each other, and introduces learnable modality type encoding, including text embedding. and image embedding ,pass and Enhance modality specificity and ultimately output a spatial semantic feature enhancement vector ; S223. In the spatiotemporal feature alignment stage, the cross-branch attention mechanism of the two-stream Transformer is used to output the temporal feature enhancement vector in each layer of the encoder in a time stream. Output feature enhancement vector with spatial semantic flow Spliced into 512-dimensional vector As input, the time stream output is the query and the spatial semantic stream output is the key-value pair, and the mutual attention score is calculated. ,in =256 is the feature vector dimension, which is consistent with the time stream branch dimension. Through cross-modal interaction, the cross-modal association of vibration time series features with infrared thermal image spatial features and maintenance log semantic features is achieved; S224. The 512-dimensional vector of the dynamic fusion stage based on the spatiotemporal feature alignment completed in step S223 , using gated attention mechanism for dynamic fusion, through the trainable matrix and bias Perform linear transformation and then generate a 256-dimensional gated value vector through the Sigmoid function , the calculation formula is: , where each element of g corresponds to the weight of the sensor time series feature, The joint weight of the corresponding text and image features; finally, the dynamic weighted formula , realize adaptive modal fusion based on real-time data characteristics, and generate joint embedding representation containing spatiotemporal correlation ,in Used for calculating the equipment health index in step S4.

6. The power plant intelligent maintenance method based on multimodal dynamic graph learning according to claim 1 is characterized in that: The method for constructing the dynamic causal graph in step S3 specifically includes: S31. Data layer: The device physical topology data is stored in a static adjacency matrix The form of sensor joint time series feature vector The sensor data is stored in a time series database. Missing values in the sensor data are filled using cubic spline interpolation. Outliers are removed based on the 3σ principle. The time window is then divided, and the sensor data is split into 1-hour sliding windows. Each window generates a sequence containing 72-hour sliding window features. ,in ; S32. Feature extraction layer: A two-layer graph convolutional neural network is used to extract features from the data output by the data layer. The calculation of each layer is based on the formula Here l is the number of layers, l takes values of 0 and 1, is the set of neighboring nodes of device i, is the trainable weight matrix, The ReLU activation function is used to combine the 128-dimensional node features output by the graph convolutional neural network with the 1024-dimensional text semantic feature vector compressed to 128 dimensions by the linear layer. The concatenation is performed to obtain a 256-dimensional vector. The fused feature vector contains the device physical topology and text semantic information, which will serve as the input for the subsequent causal inference layer. S33. Causal Inference Layer: Based on the PC algorithm, conditional independence tests are performed on sensor data and physical topology data, an initial undirected graph is constructed, and the directions of the edges are determined to obtain a directed acyclic graph as the initial causal network. 23 domain rules structured in the form of conditions and results are injected, and the network structure and edge relationships are modified to reflect the rule constraints. The initial causal network is modeled as a Bayesian network. Using the Bayesian network parameter learning algorithm, the conditional probability distribution of nodes is estimated by combining domain rules and data. The rules are converted into prior probability constraints on causal edges. The edges in the initial causal network are then tested using the d-separation criterion, and pseudo causal paths are deleted. At the same time, the GraphSAGE algorithm is used to aggregate real-time features. and historical characteristics , according to the formula Update the weights of the edges in the causal graph, where the MLP multi-layer perceptron is a 3-layer fully connected network. is the time decay factor, and are the real-time feature vectors of nodes i and j respectively, and the final output is a dynamic causal graph containing dynamic edge weights.

7. The power plant intelligent maintenance method based on multimodal dynamic graph learning according to claim 1 is characterized in that: The specific implementation of the improved proximal strategy optimization algorithm in step S4 includes: S41. State space: defining the state vector ,in , is the multimodal fusion vector The weighted sum of The SHAP value is dynamically adjusted daily to reflect the contribution of each modal feature to the health status of the equipment. θ is the preset threshold, MTBF is the mean time between failures, and spare parts inventory adequacy rate = number of available spare parts / number of maintenance requirements. S42. Action space: Contains 12 standard maintenance actions, including lubrication, tightening, replacing spare parts, and waiting. Each action is associated with a cost coefficient. ,The larger the cost coefficient value is, the higher the cost of the maintenance action. ,At the same time, these actions are also subject to some constraints, ,including the hard constraint of lubrication interval ≥ 72h, and the spare parts inventory must be greater than the ,safety threshold; S43. The reward function is designed as: , where MTTR is the mean time to repair, 、 and is the historical benchmark value. 、 and is the real-time value after executing the action, is an indicator function. If the failure probability output by the dynamic causal graph is less than 0.5 but a failure actually occurs, the function takes the value of 1, otherwise it takes the value of 0; S44. Strategy optimization: Optimizing the state sequence of the previous 72 hours through long short-term memory network Perform temporal feature extraction to obtain hidden state , and then use the proximal strategy optimization algorithm based on the current state and hidden state , through the policy network Generate action probability distribution and give priority to maintenance actions corresponding to the root cause nodes of the dynamic causal graph. Finally, use Monte Carlo tree search to simulate the decision path and calculate the advantage function through generalized advantage estimation. , taking into account the current state value and future reward discount, the formula is , where γ is the discount factor, is the state value function, which serves as the key input for policy updating and is combined with the clipping parameter 0.2 to limit the policy update amplitude.

8. The method for intelligent maintenance of a power plant based on multimodal dynamic graph learning according to claim 1, characterized in that: The triggering logic of the three-level warning mechanism in step S5 is: When the equipment health index is ≥0.7, a Level 1 warning is triggered. Based on the "routine inspection" strategy in the maintenance strategy generated in step S4, an inspection instruction is issued to the DCS system via the OPC UA protocol. This generates an equipment health report containing the failure probability and health trend, and recommends that a manual inspection be completed within three days. The instruction content is determined by the high-risk nodes output by the dynamic cause-and-effect diagram, including the inspection route and key monitoring components. When the equipment health index is 0.4 ≤ < 0.7, a secondary alert is triggered, triggering the "preventive maintenance" branch in the maintenance strategy. The system automatically checks the spare parts inventory. If the inventory sufficiency rate is less than 80%, a spare parts purchase work order is generated. A preventive maintenance work order is also generated, including the maintenance type and execution time. The equipment downtime window is locked using the OPC UA protocol. When the equipment health index falls below 0.4, a Level 3 warning is triggered, activating the "Emergency Shutdown" plan in the maintenance strategy. A hard shutdown command with the highest priority is sent to the DCS system via the OPCUA protocol. The root cause node information from the dynamic cause-and-effect graph analysis is simultaneously pushed to the maintenance terminal, initiating the emergency spare parts allocation process.

9. A power plant intelligent maintenance system based on multimodal dynamic graph learning, characterized in that: A method for implementing a power plant intelligent maintenance method based on multimodal dynamic graph learning according to any one of claims 1 to 8, comprising: Data collection layer: Deploy multi-source sensor clusters and industrial terminals to execute step S1 and collect structured sensor data, unstructured text data, image data, and physical topology data of equipment operation; Edge computing layer: deploy computing nodes, execute step S21 to pre-process the collected data and cache the feature vectors of the last 72 hours, and transmit the pre-processed feature vectors to the cloud via the OPC UA protocol 、 、 ; Cloud-based intelligent analysis layer: Integrates a multimodal fusion engine, a dynamic graph training cluster, and a reinforcement learning decision module. The multimodal fusion engine executes step S22 based on a dual-stream Transformer architecture and outputs a joint embedding representation containing spatiotemporal correlations. To the reinforcement learning decision module; the dynamic causal graph training cluster executes step S3, outputs status information including equipment failure probability, root cause node set, and causal relationship strength to the reinforcement learning decision module; the reinforcement learning decision module executes step S4, based on the output of the multimodal fusion engine The equipment status information output by the dynamic causal graph training cluster is used to generate maintenance strategies through the improved PPO algorithm, and the maintenance strategies are output to the decision execution layer; Decision execution layer: Receives the maintenance strategy output by the reinforcement learning decision module through the OPC UA protocol, triggers a three-level warning based on the strategy type, and interacts with the DCS system to execute step S5. At the same time, the decision execution layer is equipped with a maintenance knowledge base and a visual human-computer interaction interface. The maintenance knowledge base stores information related to equipment maintenance, including historical data, fault cases, and maintenance manuals, which can be used by operators for reference. The visual human-computer interaction interface has an intuitive graphical display function, supports the issuance of maintenance instructions, and can clearly display the root cause analysis results of faults, providing decision support for maintenance personnel.

Images