An unmanned inspection and state diagnosis method and system deeply integrated with an ACS

Abstract

The application discloses a kind of unmanned inspection and state diagnosis method and system deeply integrated in ACS.The method comprises: deploying fixed camera and mobile inspection robot in coal-fired power plant, performing near and far view coordinated layout, and constructing basic perception network;Obtain multi-source heterogeneous data and carry out space-time alignment and fusion, form multi-modal inspection data under unified space-time reference;Multi-modal data is input into equipment state diagnosis model pre-trained using transfer learning and domain adaptation strategy, and the output diagnosis result and abnormal positioning information;Inspection data and diagnosis result are deeply integrated into the ACS unified data platform of production control area through industrial real-time communication protocol, and are real-time fused with DCS time series data, and ACS directly triggers operation production decision according to fused data.The application breaks the isolation barrier between inspection system and control system, realizes inspection-diagnosis-decision closed loop self-healing, and significantly improves the intelligent level and operation safety of coal-fired power plant.

Description

Technical Field

[0001] This invention belongs to the field of intelligent technology for coal-fired power plants, and in particular relates to a method and system for unmanned inspection and condition diagnosis that is deeply integrated into ACS (Automatic Inspection and Diagnosis System). Background Technology

[0002] Against the backdrop of the accelerated global energy structure transition towards low-carbon development, coal-fired power plants, as stabilizers and ballast stones of energy supply, are facing multiple challenges in terms of efficiency improvement, safe operation, and intelligent upgrading. Traditional coal-fired power plant operation and maintenance models rely heavily on manual labor, resulting in the following significant defects and shortcomings: 1. Limitations of manual inspection: Traditional inspection work has a limited coverage area, especially in complex, high-altitude, or dangerous areas (such as boiler bodies, coal conveyor bridges, and booster stations), making comprehensive monitoring difficult. Furthermore, monitoring accuracy is greatly affected by personnel experience, physiological state, and environmental factors. Under harsh conditions such as low light, high dust, glare, and oil contamination, the human eye's ability to identify faults decreases sharply, leading to delayed detection of early faults such as equipment leaks, localized overheating, and electrical discharges. This can easily escalate into unplanned shutdowns and major safety accidents.

[0003] 2. Separation of Inspection and Control Systems: Existing unmanned inspection systems (such as tracked robots, drones, and fixed cameras) are typically deployed as independent subsystems, with their data, video streams, and alarm information located in the non-control area's third information zone (management information zone). The core distributed control system (DCS) is deployed in the production control zone. This physical and logical isolation prevents real-time, direct correlation analysis and fusion calculations between inspection data and DCS time-series data (such as temperature, pressure, and vibration). Inspection results are often transmitted offline to operators in the form of reports or alarm work orders, failing to form real-time closed-loop control and offering very limited support for production decision-making.

[0004] 3. Insufficient diagnostic capabilities: Most current diagnostic methods rely on a single data source (such as using only vibration data or only visible light images), making it difficult to comprehensively reflect the complex health status of equipment. Diagnostic knowledge across devices and scenarios is difficult to transfer and reuse. When equipment operating conditions change or new types of faults occur, the model's generalization ability and diagnostic accuracy will drop sharply.

[0005] Therefore, designing an unmanned inspection and condition diagnosis method that can break down system barriers, integrate multi-source heterogeneous data, operate robustly in harsh environments, and directly empower production decisions is a key technical problem that urgently needs to be solved in the current intelligent transformation of coal-fired power plants. Summary of the Invention

[0006] The present invention aims to solve the above-mentioned problems in the prior art and provide a method and system for unmanned inspection and condition diagnosis that is deeply integrated into ACS.

[0007] To achieve the above objectives, the present invention proposes the following technical solution: a method for unmanned inspection and condition diagnosis deeply integrated into ACS, characterized by comprising the following steps: Perception layer construction steps: Deploy fixed cameras and mobile inspection robots within a predetermined area of ​​the coal-fired power plant, and implement a near-field and far-field collaborative deployment strategy to construct a basic perception network covering the predetermined area.

[0008] Multi-source data fusion steps: acquire fixed visual data collected by the fixed camera, as well as mobile visual data, infrared thermal imaging data, partial discharge detection data and robot pose data collected by the mobile inspection robot during autonomous movement; perform spatiotemporal alignment and fusion of the acquired multi-source heterogeneous data to form multimodal inspection data under a unified spatiotemporal reference.

[0009] Multimodal state diagnosis steps: The multimodal inspection data is input into a pre-trained equipment state diagnosis model, a high-discrimination feature vector is extracted through a deep feature extraction network, and state recognition is performed based on the feature vector, outputting the equipment state diagnosis results and anomaly location information; wherein, the equipment state diagnosis model adopts a transfer learning and domain adaptation strategy for cross-working-condition generalization training.

[0010] Deep integration and decision-making closed-loop steps: The generated multimodal inspection data and status diagnosis results are deeply integrated and published to the unified data platform of the Autonomous Decision Intelligent Control System (ACS) in Production Control Zone 1 through industrial real-time communication protocol, and fused with the time-series data of the Distributed Control System (DCS) in real time; the ACS directly triggers production decision-making based on the fused data, forming a closed-loop control process of inspection-diagnosis-decision linkage.

[0011] Furthermore, the near-field and far-field collaborative deployment strategy includes: for the high-precision monitoring needs of key equipment, deploying high-resolution fixed cameras within a preset threshold distance to the equipment to perform near-field detailed inspection tasks; for the security situation monitoring needs of a large area, deploying fixed cameras with optical zoom capabilities at high points to perform far-field general inspection tasks; and dynamically scheduling the near-field and / or far-field fixed cameras based on the real-time pose and preset inspection path of the mobile inspection robot to jointly re-inspect and confirm the abnormal areas identified by the mobile inspection robot from multiple angles.

[0012] Furthermore, the spatiotemporal alignment and fusion of the acquired multi-source heterogeneous data specifically includes: Time alignment steps: A global clock synchronization mechanism based on IEEE 1588 Precise Time Protocol (PTP) or Network Time Protocol (NTP) is used to assign a unified timestamp to each frame of visual data, infrared data, partial discharge data, and pose data; for asynchronously sampled data streams, interpolation or timestamp nearest neighbor matching algorithms are used for time registration.

[0013] Spatial alignment steps: Establish a unified global coordinate system, obtain the intrinsic and extrinsic parameters of the fixed camera and the odometer and inertial measurement unit (IMU) parameters of the mobile inspection robot through calibration; based on the robot's Simultaneous Localization and Mapping (SLAM) technology, convert the data collected by the mobile sensor to the global coordinate system in real time to achieve data spatial registration with the fixed camera's perspective.

[0014] Fusion steps: The temporally and spatially aligned images, infrared thermal images, partial discharge signals, and pose data are fused at the pixel level or feature level to generate unified, spatiotemporally labeled multimodal inspection data.

[0015] Furthermore, the multimodal state diagnostic model includes: Multimodal feature extraction subnetwork: includes a convolutional neural network (CNN) branch for processing visible light images, a thermal feature extraction branch for processing infrared thermal images, and a temporal neural network branch for processing partial discharge signals.

[0016] Cross-modal attention fusion module: Employs a multi-head attention mechanism to adaptively weight and fuse the feature vectors output by the multi-modal feature extraction sub-network to enhance key features related to equipment faults and suppress background and noise interference.

[0017] State classification and anomaly localization sub-network: Based on the fused feature vector, it outputs the probability distribution of the health status of the device under different operating conditions, and combines the coordinate mapping relationship in the spatial alignment step to generate a heat map or bounding box of the abnormal area on the original image.

[0018] Furthermore, the method of employing transfer learning and domain adaptation strategies for cross-condition generalization training specifically includes: The device status diagnosis model is pre-trained on source domain data, which includes historical fault data and simulation data.

[0019] Domain Adversarial Neural Network (DANN) or Correlation Alignment (CORAL) algorithm is used to minimize the feature distribution difference between the source domain and the target domain (the current actual operating conditions of the power plant), so that the model learns the domain-invariant feature representation.

[0020] By using a small number of labeled samples in the target domain, the pre-trained model is fine-tuned to adapt to individual differences in specific devices and changes in real-time operating conditions.

[0021] Furthermore, in the deep integration and decision-making closed-loop step, the ACS directly triggers production decisions based on the fused data, specifically including: The ACS's intelligent decision engine performs association rule mining or causal inference on the status diagnosis results (including fault type, confidence level, and location information) and the DCS time series data (including temperature, pressure, flow rate, and vibration amplitude).

[0022] When the diagnostic results and DCS parameters together meet the preset fault triggering logic, the ACS automatically calls the preset fault handling strategy library, generates and sends closed-loop control commands to the DCS actuator to achieve fault self-healing.

[0023] If the status diagnosis result indicates an intermittent or complex fault requiring manual intervention, a three-dimensional visual alarm card containing the fault location, diagnostic basis, and suggested operation steps is generated on the operator station through the intelligent human-machine interaction engine integrated into the ACS, thereby realizing human-machine collaborative decision-making.

[0024] This invention also provides an unmanned inspection and condition diagnosis system deeply integrated into ACS, used to implement the method described in any of the above-mentioned embodiments, characterized in that it includes: The perception layer system includes a fixed camera group deployed according to a near-far field collaborative deployment strategy, and a mobile inspection robot equipped with a visible light camera, an infrared thermal imager, a partial discharge sensor, and a pose sensor.

[0025] Data fusion and diagnostic system: Deployed on edge computing nodes or cloud servers, including a data spatiotemporal alignment module, a multimodal feature extraction and fusion module, and a device status diagnostic module based on transfer learning.

[0026] Deep integration and decision-making system: namely the autonomous decision-making intelligent control system (ACS), deployed in production control zone 1, including a unified data platform, an intelligent decision engine and a human-machine interaction engine; the unified data platform is used to receive and integrate the multimodal inspection data and DCS time series data output by the data fusion and diagnosis system in real time.

[0027] Furthermore, it also includes an edge-cloud collaborative architecture: The edge computing nodes are deployed on the field side to perform data preprocessing, spatiotemporal alignment, and rapid detection and preliminary diagnosis of sudden anomalies, which have high real-time requirements.

[0028] The cloud server is used to perform computationally intensive tasks, including in-depth diagnosis of complex faults, incremental training and updating of equipment status diagnosis models, and cross-device fault knowledge transfer.

[0029] The edge computing nodes and cloud servers synchronize data and update models via industrial 5G or fiber optic networks.

[0030] Furthermore, the intelligent decision engine has a built-in decision knowledge base based on knowledge graphs and curve morphology topology; The decision knowledge base is used to store historical fault cases, handling strategies, and the correlation between DCS time-series characteristics and fault phenomena.

[0031] The curve morphology topology is used to decouple and encode the shape features (including trends, peaks, and periods) of DCS time series data, so as to perform efficient similarity matching and retrieval with the multimodal inspection data, thereby providing an interpretable decision basis for the current fault.

[0032] In addition, the present invention provides a computer-readable storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of the unmanned inspection and condition diagnosis method deeply integrated into ACS as described above.

[0033] Beneficial effects: (1) Improve inspection coverage and efficiency: By deploying fixed cameras and mobile inspection robots in a near-far view coordinated manner, combined with intelligent path planning, high-precision monitoring of key equipment is ensured, and low-cost coverage of a large area is achieved, which solves the contradiction between coverage, accuracy and cost in traditional solutions.

[0034] (2) Enhance the robustness of recognition in harsh environments: By fusing multimodal data such as visible light, infrared, and partial discharge, and using the deep attention mechanism to adaptively enhance key features, the problem of traditional visual algorithms being prone to failure under harsh working conditions such as low light, high dust, and oil pollution is effectively overcome, and the ability to identify weak defects and early faults of equipment is significantly improved.

[0035] (3) Improve diagnostic accuracy and generalization ability: Multimodal deep feature extraction and cross-modal fusion were adopted to construct a highly discriminative equipment state feature vector. Combined with transfer learning and domain adaptation technology, the diagnostic model can quickly adapt to different equipment and different working conditions, and has good generalization diagnostic ability for occasional faults and unseen fault types, thus enhancing the effect of cross-plant fault knowledge transfer.

[0036] (4) Breaking down system barriers and enabling real-time decision-making: The unmanned inspection system is deeply integrated into the ACS production control zone, achieving real-time fusion with DCS time-series data. This is the first time in the industry that such deep integration has been achieved, so that inspection results are no longer offline reports, but serve as key real-time input parameters that directly participate in the ACS's operational decisions. The system can automatically trigger control strategies based on diagnostic results, achieving self-healing of more than 85% of typical scenario faults, providing core technical support for ultimately achieving "single-person duty" and "near-zero intervention" operation of unit units. Attached Figure Description

[0037] Figure 1 This is an overall architecture diagram of the unmanned inspection and condition diagnosis method deeply integrated into ACS provided in the embodiments of the present invention.

[0038] Figure 2 This is a schematic diagram illustrating the coordinated deployment of fixed cameras for near and far views and the coordinated mobile inspection provided in an embodiment of the present invention.

[0039] Figure 3 A flowchart of spatiotemporal alignment and fusion of multi-sensor data provided in an embodiment of the present invention.

[0040] Figure 4 A flowchart for multi-modal state diagnosis, including visible light, infrared, and partial discharge, provided for embodiments of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] Example 1 The unmanned inspection and condition diagnosis method deeply integrated into ACS proposed in this invention will systematically explain its technical principles and implementation logic in a typical coal-fired power plant scenario, from four core aspects: perception layer construction, multi-source data fusion, condition diagnosis model construction and training, and deep integration and decision-making closed loop.

[0043] In key areas of coal-fired power plants, such as boiler rooms, coal conveyor bridges, substations, and turbine rooms, fixed cameras and mobile inspection robots are deployed first. Fixed cameras employ a combined close-up and long-range deployment strategy. For example, high-resolution industrial-grade fixed cameras are deployed at distances of 1.5 to 3 meters from key equipment such as boiler tubes, turbine bearings, and coal conveyor belt rollers to perform close-up inspections, capturing subtle anomalies such as surface cracks, oil leaks, and localized overheating. Simultaneously, panoramic cameras with optical zoom capabilities are deployed at high points in the plant, such as the boiler roof and high platforms of the coal conveyor bridge, to perform long-range inspections, monitoring the overall safety situation over a large area, such as personnel intrusion, equipment displacement, and ash / dust accumulation.

[0044] The mobile inspection robot is equipped with a visible light camera, an infrared thermal imager, a partial discharge sensor (such as an ultrasonic or ground wave sensor), and a pose sensor (including an odometer and an IMU). The robot moves autonomously along a preset inspection path, collecting multimodal data of the equipment along the way in real time. More importantly, the system supports dynamic linkage: when the mobile robot identifies an abnormal temperature or discharge characteristics in a certain area of ​​equipment during the inspection, the system will automatically dispatch one or more near-field or far-field fixed cameras in the vicinity of the area based on the robot's real-time pose information to conduct a multi-angle joint re-inspection of the abnormal area, thereby achieving secondary confirmation and precise spatial positioning of suspected faults.

[0045] Since fixed cameras and mobile robots collect data from different sources, use different coordinate systems, and have different sampling frequencies, strict time and space alignment is necessary.

[0046] For time alignment, the system employs a global clock synchronization mechanism, such as the IEEE 1588 Precision Time Protocol (PTP) or Network Time Protocol (NTP). All sensor data is timestamped uniformly upon generation. For data streams with different sampling frequencies, such as 30 frames / second for visible light images, 10 frames / second for infrared thermal images, and 1kHz for partial discharge signals, the system uses interpolation or a timestamp nearest neighbor matching algorithm for time registration, ensuring that multimodal data acquired within the same time window can be correctly correlated.

[0047] For spatial alignment, the system first establishes a unified global 3D coordinate system, for example, using a fixed point at the 0-meter level of a coal-fired power plant boiler as the origin. The intrinsic parameters (focal length, distortion parameters) and extrinsic parameters (position, orientation) of the fixed camera are obtained through offline calibration, while the odometry and IMU parameters of the mobile robot are also calibrated. During robot operation, Simultaneous Localization and Mapping (SLAM) technology is used to estimate its pose in the global coordinate system in real time, thereby converting the robot's collected mobile visual data, infrared thermal images, partial discharge signals, etc., to the global coordinate system in real time, achieving spatial registration with the data from the fixed camera.

[0048] After completing spatiotemporal alignment, the system performs pixel-level or feature-level fusion of visible light images, infrared thermal images, partial discharge signals, and pose data to generate unified, spatiotemporally labeled multimodal inspection data. For example, the high-temperature areas of the infrared thermal image are superimposed on the visible light image to form a hot spot overlay map; or the energy distribution of the partial discharge signal is mapped to the three-dimensional spatial position of the device to form a discharge hotspot map.

[0049] The diagnostic model is the core intelligent component of the method of this invention. It adopts a deep learning architecture and includes three sub-networks: The multimodal feature extraction subnetwork comprises three parallel branches. The visible light image branch uses a pre-trained convolutional neural network (such as ResNet or EfficientNet) to extract features such as surface texture, shape, and color of the device; the infrared thermal image branch uses a thermal feature extraction network to extract features such as temperature distribution, thermal gradient, and hotspot regions; and the partial discharge signal branch uses a temporal neural network (such as LSTM or Transformer) to extract temporal features such as the amplitude, phase, and repetition rate of the discharge pulse.

[0050] Cross-modal attention fusion module: Employing a multi-head attention mechanism, this module adaptively weights and fuses the feature vectors output from the three branches. It automatically enhances modal features highly correlated with fault types while suppressing background noise or non-contributing modes. For example, in low-light environments where visible light image quality degrades, the model automatically reduces the weight of visible light features while enhancing the contributions of infrared thermal images and partial discharge features.

[0051] The status classification and anomaly localization subnetwork, based on the fused feature vectors, outputs the probability distribution of the device's health status under different operating conditions, such as normal, warning, severe, and critical. Simultaneously, by combining the coordinate mapping relationship in spatial alignment, it generates heatmaps or bounding boxes of abnormal regions on the original image, achieving pixel-level localization of fault locations.

[0052] In terms of model training, this invention employs a transfer learning and domain adaptation strategy. First, the model is pre-trained on source domain data, including historical fault data and simulation data, such as millions of labeled samples from multiple power plants. Then, a Domain Adversarial Neural Network (DANN) or Correlation Alignment (CORAL) algorithm is used to minimize the feature distribution differences between the source domain and the target domain (the current actual operating conditions of the power plant), enabling the model to learn domain-invariant feature representations. Finally, the pre-trained model is fine-tuned using a small number of labeled samples from the target domain (e.g., only 10 to 20 samples per type of fault) to adapt to individual differences in specific equipment and real-time operating condition changes.

[0053] Traditional unmanned inspection systems typically send alarm information to the management information area, physically isolated from the DCS system in the production control area, thus failing to form a closed-loop control. This invention breaks down this barrier. Multimodal inspection data and diagnostic results are deeply integrated through industrial real-time communication protocols (such as OPCUA or MQTToverTSN) and published to the unified data platform of the Autonomous Decision-Making Intelligent Control System (ACS) in the production control area.

[0054] Within the ACS platform, the intelligent decision engine performs association rule mining or causal inference on the status diagnosis results (including fault type, confidence level, and location information) and DCS time-series data (such as temperature, pressure, flow rate, and vibration amplitude). For example, when the diagnostic model detects that the infrared thermogram of a coal conveyor belt roller shows a continuous rise in temperature, and at the same time the current data of the motor in the DCS system shows periodic fluctuations, and the two together meet the preset "overheating, overload" fault triggering logic, ACS will automatically call the pre-set fault handling strategy library, generate closed-loop control commands, and issue them to the DCS actuators, such as reducing the belt running speed or starting the backup cooling system, to achieve fault self-healing.

[0055] If the diagnostic results indicate an intermittent or complex fault requiring manual intervention, such as a rare high-frequency discharge pattern detected in partial discharge detection with a confidence level below the threshold, the system generates a 3D visual alarm card on the operator station through the intelligent human-machine interaction engine integrated into the ACS. This card includes the fault location, diagnostic basis (such as characteristic heatmaps and partial discharge waveforms), and suggested operating steps, enabling human-machine collaborative decision-making.

[0056] Example 2 This embodiment will be described in conjunction with the appendix to the instruction manual. Figures 1 to 4 This paper takes the boiler heating surface of a large-scale ultra-supercritical coal-fired power plant as an example to fully illustrate the actual operation process of the unmanned inspection and condition diagnosis method deeply integrated into the ACS proposed in this invention. The boiler heating surface is subjected to high temperature, high pressure, corrosive atmosphere, and fly ash erosion for extended periods, making it one of the areas with the highest incidence of failures and the most severe consequences. Traditional manual inspections are almost impossible to access the furnace interior while the boiler is running, and post-shutdown inspections are lengthy and costly, making it difficult to provide early warnings for accidents such as tube ruptures and leaks. This embodiment fully demonstrates the technical advantages of this invention under extreme operating conditions.

[0057] like Figure 1 As shown, the overall architecture of this invention is divided into a perception layer, a data fusion and diagnosis layer, and a deep integration and decision-making layer. In this embodiment, the deployment of the perception layer strictly follows... Figure 2 The near-field and far-field coordinated deployment strategy is shown.

[0058] Fixed camera deployment: High-temperature resistant, dust-proof, high-resolution fixed cameras will be deployed near manholes and observation holes on the outer side of the boiler water-cooled walls, as well as at wall penetration points of the superheater and reheater tube panels, within a distance of approximately one meter from the tube wall, according to close-range detailed inspection requirements. The number of cameras will be determined based on site needs. These close-range cameras will be used to capture minute defects such as oxide scale peeling, creep bulging, and cracks on the tube wall surface. Simultaneously, spherical cameras equipped with optical zoom and infrared thermography functions will be deployed at high points on different boiler platforms for long-range general inspection of macroscopic anomalies such as overall deformation, ash collapse, and coking of the tube panels over a large area.

[0059] Mobile Inspection Robots: Several tracked mobile inspection robots are deployed in the external inspection channels of the boiler body. Each robot is equipped with a visible light camera, an infrared thermal imager, an ultrasonic partial discharge sensor, and a lidar and pose sensor for SLAM positioning. The robots automatically travel along a preset path and complete the inspection of all visible external pipe sections of the boiler body according to a set schedule.

[0060] Dynamic collaborative operation: When a mobile robot is inspecting a platform, its infrared thermal imager detects an abnormal surface temperature in a section of the water-cooled wall. The system immediately and automatically dispatches nearby fixed close-up cameras and a distant spherical camera, based on the robot's real-time pose, to conduct a multi-angle joint re-inspection of the abnormal area. The close-up camera clearly captures minute cracks on the surface of the water-cooled wall tube, while the distant camera confirms that there is no large-area coking interference in the area. This collaborative mechanism significantly improves the reliability of the diagnosis.

[0061] Figure 3 The data alignment and fusion process is illustrated in detail. In this embodiment, data from various sensors are processed through the following steps: Time Alignment: The entire system employs a high-precision clock synchronization protocol, such as IEEE 1588 PTP, to achieve sub-millisecond synchronization. Fixed cameras acquire images at a standard frame rate, the mobile robot's infrared thermal imager acquires thermal images at a slightly lower frame rate, and the partial discharge sensor acquires signals at a higher sampling rate. For asynchronous data, the system uses an interpolation algorithm to align the timestamps of the infrared and partial discharge data to each frame of the visible light image, ensuring that multimodal data acquired at the same time point can be accurately correlated. For example, if the robot detects a temperature peak at a certain moment, and the partial discharge pulse is also concentrated within a small time window around that moment, it determines that both originate from the same event.

[0062] Spatial Alignment: First, a global 3D coordinate system is established with a fixed point on the boiler as the origin. Through offline calibration, the intrinsic and extrinsic parameters of all fixed cameras are obtained. For the mobile robot, SLAM technology is used to construct a real-time map of the boiler's external environment, and the robot's pose in the global coordinate system is estimated by combining this data with LiDAR data. Then, each pixel in the infrared thermal image and visible light image carried by the robot is projected onto the global coordinate system through a coordinate transformation matrix. Simultaneously, the detection results from the partial discharge sensor are combined with the robot's pose to calculate the approximate position of the discharge sound source in the global coordinate system.

[0063] Data Fusion: After completing spatiotemporal alignment, the system performs feature-level fusion. Texture feature vectors such as cracks and wear are extracted from visible light images; thermal feature vectors such as temperature gradients and hotspot morphology are extracted from infrared thermal images; and feature vectors such as pulse amplitude, repetition rate, and phase distribution are extracted from partial discharge signals. These three vectors are then concatenated into a high-dimensional feature vector and input into the subsequent cross-modal attention module. The fused data is then labeled with a unified spatiotemporal tag, forming a standardized multimodal inspection data package.

[0064] Figure 4 The specific structure of the diagnostic model is shown. In this embodiment, the model operates as follows, targeting high-temperature corrosion and creep fatigue failure of the boiler water-cooled wall: Visible light branch: A pre-trained deep convolutional neural network is used to extract the texture and shape features of crack and oxide peeling areas, and output high-dimensional feature vectors.

[0065] Infrared branch: Construct a thermal feature extraction network containing multiple convolutional layers and fully connected layers. Input the temperature matrix of the infrared thermal image and output the extreme values, average values, thermal gradients and other features of the hot spot areas, which are integrated into a feature vector.

[0066] Partial discharge branch: Uses a time-series neural network to process the time sequence of partial discharge signals, extracts features such as pulse peak value, pulse interval, and frequency distribution, and outputs feature vectors.

[0067] Cross-modal attention fusion: The multi-head attention module adaptively weights the feature vectors of the three branches mentioned above. For example, under the background of high-temperature radiation inside the boiler, infrared features may be interfered with by the furnace flame, but the partial discharge signal attenuates less and is stable when it is conducted on the high-temperature metal surface. Therefore, the attention module will automatically assign higher weights to the partial discharge features, while appropriately reducing the weights of infrared and visible light features.

[0068] State classification and anomaly localization: The classification subnetwork consists of a fully connected layer and a Softmax layer, outputting the probability distribution of the device under different health states, such as normal, high-temperature corrosion warning, creep cracking, and leakage hazard. Simultaneously, the localization subnetwork, combined with spatially aligned coordinate mapping, generates heat maps or bounding boxes for anomaly areas on the original image. For example, it marks the crack location with a red bounding box on a visible light image of a water-cooled wall tube and estimates the physical size of the crack.

[0069] Transfer Learning and Domain Adaptation: This power plant is a newly commissioned unit with very limited historical fault data. To address the small sample size problem, the system first loads a pre-trained model onto source domain data, which includes historical fault data from other units and simulated defect samples. Then, a domain adversarial neural network or relevance alignment algorithm is used to minimize the feature distribution differences between the source domain and the target domain. Finally, the pre-trained model is fine-tuned using a small number of existing labeled samples from this power plant to adapt it to the equipment characteristics and real-time operating conditions of this unit.

[0070] The diagnostic results were published to the ACS unified data platform in Production Control Zone 1 via the industrial real-time communication protocol. The ACS immediately correlated the relevant parameters of the water-cooled wall area in the DCS system: for example, although the average pipe wall temperature in this area was within the normal range over a period of time, the fluctuation amplitude was significantly increased; the working fluid flow rate in the water-cooled wall circulation loop showed a slight decrease; and the flue gas temperature measurement points in adjacent areas showed local abnormal increases.

[0071] ACS's intelligent decision engine incorporates a knowledge graph and curve morphology topology to perform association rule matching between the aforementioned multimodal diagnostic results and DCS time-series data. The knowledge graph stores typical association patterns such as "water-cooled wall creep cracking leading to increased wall temperature fluctuations, decreased working fluid flow, localized high temperatures in infrared thermography, and visible light cracks." Curve morphology topology encodes the shape of the DCS wall temperature time-series curve, revealing a high degree of similarity between its fluctuation pattern and historical cracking cases. A comprehensive assessment indicates a high risk level.

[0072] Because this fault (early crack in the water-cooled wall) could develop into a tube rupture within a short period if not addressed promptly, leading to unplanned shutdowns and equipment damage, the ACS automatically triggers the "load reduction and stress relief" strategy from its pre-set fault handling strategy library. The system issues closed-loop control commands to the DCS: for example, reducing the unit load to a safe level to decrease thermal stress, adjusting the burner angle and air distribution to reduce the heat load in that area, and simultaneously generating a mandatory alarm card at the operator station, prompting operators to request a shutdown for maintenance in the near future. The system also automatically dispatches work orders to the maintenance department, including the precise location of the crack, suggested repair solutions, and a list of required spare parts.

[0073] From the moment the robot detects a crack to the ACS automatically reducing its load, the entire process is very short, much faster than a manual response. If the diagnostic confidence is low (e.g., only a minor abnormality is detected without a clear crack), the ACS will not automatically reduce its load. Instead, a 3D visual alarm card will pop up on the screen via the human-machine interface engine, which will be manually adjusted by the operators after confirmation. This hierarchical decision-making mechanism ensures both rapid and automatic handling of major hidden dangers and avoids erroneous actions.

[0074] This embodiment also utilizes an edge-cloud collaborative architecture. Edge computing nodes are deployed in the boiler electronics room to perform real-time data preprocessing, spatiotemporal alignment, and preliminary diagnosis. When a high-confidence critical fault is detected, it is immediately reported to the ACS. Simultaneously, the entire diagnostic process data is anonymized and uploaded to the cloud server. The cloud server periodically performs incremental training, adding newly confirmed fault cases to the source domain dataset and updating the diagnostic model. The updated model is then distributed to the edge nodes via the industrial network.

[0075] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for unmanned inspection and condition diagnosis deeply integrated into ACS, characterized in that, Includes the following steps: Fixed cameras and mobile inspection robots are deployed within a predetermined area of ​​a coal-fired power plant, and a strategy of coordinating close-up and long-range deployment is implemented to construct a basic perception network covering the predetermined area. The fixed visual data collected by the fixed camera, as well as the mobile visual data, infrared thermal imaging data, partial discharge detection data and robot pose data collected by the mobile inspection robot during autonomous movement, are obtained to obtain multi-source heterogeneous data; the obtained multi-source heterogeneous data are spatiotemporally aligned and fused to form multimodal inspection data under a unified spatiotemporal reference. The multimodal inspection data is input into a pre-trained equipment condition diagnosis model. A deep feature extraction network is used to extract high-discrimination feature vectors, and condition recognition is performed based on the high-discrimination feature vectors to output the equipment condition diagnosis results and anomaly location information. The equipment condition diagnosis model adopts transfer learning and domain adaptation strategies for cross-working-condition generalization training. The generated multimodal inspection data and status diagnosis results are deeply integrated and published to the unified data platform of the Autonomous Decision Intelligent Control System (ACS) in Production Control Zone 1 through industrial real-time communication protocol. The ACS is then fused in real time with the time-series data of the Distributed Control System (DCS). The ACS directly triggers production decisions based on the fused data.

2. The method according to claim 1, characterized in that, The coordinated deployment strategy for near-field and far-field views includes: To meet the high-precision monitoring needs of key equipment, high-resolution fixed cameras are deployed within a preset threshold distance from the equipment to perform close-range detailed inspection tasks. To meet the security situation monitoring needs of large areas, fixed cameras with optical zoom capabilities are deployed at high points to carry out long-range survey tasks. Based on the real-time pose and preset inspection path of the mobile inspection robot, the near-field and / or far-field fixed cameras are dynamically scheduled to jointly re-inspect and confirm the abnormal areas identified by the mobile inspection robot from multiple angles.

3. The method according to claim 1, characterized in that, The spatiotemporal alignment and fusion of the acquired multi-source heterogeneous data specifically includes: A global clock synchronization mechanism based on IEEE 1588 Precision Time Protocol (PTP) or Network Time Protocol (NTP) is adopted to assign a unified timestamp to each frame of visual data, infrared data, partial discharge data, and pose data; for asynchronously sampled data streams, interpolation or timestamp nearest neighbor matching algorithms are used for time registration. A unified global coordinate system is established, and the intrinsic and extrinsic parameters of the fixed camera and the odometer and IMU parameters of the mobile inspection robot are obtained through calibration. Based on the real-time localization and mapping (SLAM) technology of the mobile inspection robot, the data collected by the mobile sensor is converted to the global coordinate system in real time to achieve data spatial registration with the fixed camera's perspective. The time-aligned and space-aligned images, infrared thermal images, partial discharge signals, and pose data are fused at the pixel level or feature level to generate unified, spatiotemporally labeled multimodal inspection data.

4. The method according to claim 1, characterized in that, The multimodal state diagnostic model includes: The multimodal feature extraction subnetwork includes a convolutional neural network branch for processing visible light images, a thermal feature extraction branch for processing infrared thermal images, and a temporal neural network branch for processing partial discharge signals. The cross-modal attention fusion module employs a multi-head attention mechanism to adaptively weight and fuse the feature vectors output by the multi-modal feature extraction sub-network, thereby enhancing key features related to equipment faults and suppressing background and noise interference. The state classification and anomaly localization subnetwork, based on the fused feature vector, outputs the probability distribution of the health status of the device under different operating conditions, and, combined with the coordinate mapping relationship in the spatial alignment step, generates a heat map or bounding box of the abnormal region on the original image.

5. The method according to claim 1, characterized in that, The method of employing transfer learning and domain adaptation strategies for cross-condition generalization training specifically includes: The device status diagnosis model is pre-trained on source domain data, which includes historical fault data and simulation data. By employing Domain Adversarial Neural Network (DANN) or Correlation Alignment (CORAL) algorithm, the feature distribution difference between the source and target domains is minimized, enabling the device condition diagnosis model to learn domain-invariant feature representations. By using a small number of labeled samples in the target domain, the pre-trained model is fine-tuned to adapt to individual differences in specific equipment and changes in real-time operating conditions.

6. The method according to claim 1, characterized in that, The ACS directly triggers operational production decisions based on the fused data, specifically including: The intelligent decision engine of the ACS performs association rule mining or causal inference on the status diagnosis results and the DCS time series data; the status diagnosis results include fault type, confidence level, and location information, and the DCS time series data includes temperature, pressure, flow rate, and vibration amplitude. When the diagnostic results and DCS parameters together meet the preset fault triggering logic, the ACS automatically calls the preset fault handling strategy library, generates and sends closed-loop control commands to the DCS actuator to achieve fault self-healing. If the status diagnosis result indicates an intermittent or complex fault requiring manual intervention, a three-dimensional visual alarm card containing the fault location, diagnostic basis, and suggested operation steps is generated on the operator station through the intelligent human-machine interaction engine integrated into the ACS, thereby realizing human-machine collaborative decision-making.

7. A system for unmanned inspection and condition diagnosis deeply integrated into ACS, used to implement the method according to any one of claims 1 to 6, characterized in that, include: The perception layer module includes a fixed camera group deployed according to a near-far collaborative deployment strategy, and a mobile inspection robot equipped with a visible light camera, an infrared thermal imager, a partial discharge sensor, and a pose sensor. The data fusion and diagnostic module, deployed on edge computing nodes or cloud servers, includes a data spatiotemporal alignment module, a multimodal feature extraction and fusion module, and a device status diagnostic module based on transfer learning. The deep integration and decision-making module, namely the Autonomous Decision-Making Intelligent Control System (ACS), is deployed in Production Control Zone 1 and includes a unified data platform, an intelligent decision-making engine, and a human-computer interaction engine. The unified data platform is used to receive and integrate the multimodal inspection data and DCS time-series data output by the data fusion and diagnostic system in real time.

8. The system according to claim 7, characterized in that, It also includes edge-cloud collaborative architecture: The edge computing nodes are deployed on the field side to perform data preprocessing, spatiotemporal alignment, and rapid detection and preliminary diagnosis of sudden anomalies with high real-time requirements. The cloud server is used to perform computationally intensive tasks, including deep diagnosis of complex faults, incremental training and updating of equipment status diagnosis models, and cross-device fault knowledge transfer. The edge computing nodes and cloud servers synchronize data and update models via industrial 5G or fiber optic networks.

9. The system according to claim 7, characterized in that, The intelligent decision engine has a built-in decision knowledge base based on knowledge graphs and curve morphology topology. The decision knowledge base is used to store historical fault cases, handling strategies, and the correlation between DCS time-series characteristics and fault phenomena. The curve morphology topology is used to decouple and encode the shape features of DCS time series data, so as to perform efficient similarity matching and retrieval with the multimodal inspection data, and provide an interpretable decision basis for the current fault; wherein, the shape features of the DCS time series data include trends, peaks, and periods.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the unmanned inspection and condition diagnosis method deeply integrated into the ACS as described in any one of claims 1 to 6.

Images