A hydropower station primary equipment intelligent inspection system and method based on multi-modal anomaly detection

Abstract

The application discloses a kind of based on multimodal anomaly detection's hydroelectric station primary equipment intelligent inspection system and method, system includes data acquisition platform, multimodal anomaly detection model connects data acquisition platform, utilize multilevel convolutional neural network to visible light image and infrared image are carried out depth feature extraction, and environmental data are fused analysis, intelligent inspection platform, connect multimodal anomaly detection model, execute real-time monitoring, anomaly detection early warning and fault diagnosis and data analysis, provide remote control and management.The appearance damage, temperature anomaly, crack and corrosion of equipment, etc. Key issues are efficiently monitored by combining visible light and infrared imaging technology. Through multimodal data fusion, not only can improve the accuracy of fault detection, but also can improve the identification ability of equipment comprehensive anomaly, so as to effectively prevent and handle the common problems in hydroelectric station primary equipment, improve the intelligent level and stability of inspection system.

Description

Technical Field

[0001] This invention belongs to the field of intelligent inspection technology, and in particular relates to an intelligent inspection system and method for primary equipment in hydropower stations based on multimodal anomaly detection. Background Technology

[0002] As a critical component of the power system, the stable operation of primary equipment in hydropower stations directly impacts the safety of the power grid and the reliable transmission of electricity. During routine inspections, key monitoring of primary equipment in hydropower stations should focus on issues such as overheating, damage, corrosion, cracks, poor contact, and electrical faults.

[0003] Traditional inspection methods primarily rely on manual inspection and video surveillance, typically using visible light imaging technology to monitor equipment status. However, traditional visible light monitoring systems are significantly less effective in low light, nighttime, or inclement weather conditions, making it difficult to effectively detect subtle equipment faults or anomalies. Therefore, infrared thermal imaging technology has been introduced to detect temperature anomalies in equipment, such as overheating and malfunctions, which is particularly effective in low-light environments. However, some existing intelligent inspection systems generally apply infrared images separately from visible light images, lacking fusion analysis of the two and failing to effectively integrate the advantages of both image data, thus limiting the comprehensiveness and accuracy of fault detection. The shortcomings and deficiencies of existing technologies are mainly reflected in three aspects: First, most existing hydropower station inspection systems rely on a single monitoring technology, typically using only visible light or infrared imaging for equipment monitoring. Visible light monitoring provides clear images under normal lighting conditions, but its effectiveness drops significantly in low light or inclement weather, making comprehensive equipment monitoring impossible. While infrared imaging can operate in low-light environments and detect temperature anomalies, it only provides temperature data and lacks comprehensive identification of equipment shape and condition. Current inspection systems generally fail to achieve fusion analysis of visible light and infrared image data, resulting in an inability to fully utilize the advantages of both, providing accurate and comprehensive detection results, and failing to effectively provide early warnings of equipment failures.

[0004] Second, while existing intelligent inspection systems incorporate image processing and artificial intelligence algorithms to some extent, most still rely on manual intervention for fault diagnosis and handling. Although some intelligent algorithms can perform image analysis, these algorithms are mostly based on processing single image sources, lacking deep fusion of different image information and intelligent decision-making. This results in low inspection efficiency and susceptibility to human experience and judgment biases, thus reducing the system's automation and intelligence levels.

[0005] Third, existing inspection systems have poor adaptability to complex environments. For example, in extreme weather, strong sunlight, or high or low temperature conditions, existing single-image monitoring systems are easily affected by environmental factors, leading to unstable monitoring results and failing to ensure continuous equipment monitoring. These problems prevent existing systems from fully realizing their potential in variable and complex working environments, affecting the timely detection and handling of equipment failures, and leaving considerable room for improvement.

[0006] In summary, existing technologies suffer from a series of shortcomings, such as reliance on single image data, lack of multimodal data fusion, high dependence on manual intervention, and poor environmental adaptability. These problems limit the intelligence level and application effect of hydropower station inspection systems. Therefore, it is necessary to propose an intelligent inspection system and method for primary equipment in hydropower stations based on multimodal anomaly detection to solve the above problems. Summary of the Invention

[0007] The technical problem this invention aims to solve is to provide an intelligent inspection system and method for primary equipment in hydropower stations based on multimodal anomaly detection. This system addresses the shortcomings of existing hydropower station primary equipment inspection systems, such as poor inspection effectiveness, low efficiency, low intelligence, and inability to operate stably in complex environments. By combining visible light and infrared imaging technologies, it efficiently monitors key issues such as equipment surface damage, temperature anomalies, cracks, and corrosion. Through multimodal data fusion, it not only improves the accuracy of fault detection but also enhances the ability to identify comprehensive equipment anomalies, thereby effectively preventing and handling common problems in hydropower station primary equipment and improving the intelligence and stability of the inspection system.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A smart inspection system for primary equipment in a hydropower station based on multimodal anomaly detection includes: Data acquisition platform: consists of visible light cameras, infrared thermal imaging cameras, and environmental monitoring sensors; the environmental monitoring sensors include temperature, humidity, air pressure, and light sensors; A multimodal anomaly detection model is connected to a data acquisition platform. It uses a multi-level convolutional neural network to extract deep features from visible light and infrared images and performs fusion analysis on environmental data. The intelligent inspection platform connects to a multimodal anomaly detection model, performs real-time monitoring, anomaly detection and early warning, fault diagnosis and data analysis, and provides remote control and management.

[0009] Preferably, the multimodal anomaly detection model includes: Data preprocessing module: Based on the resolution differences, noise, and illumination changes between visible light and infrared images, preprocess the data from the infrared camera and the visible light camera respectively, and output standardized images; The feature extraction encoder includes a visible light encoder and an infrared encoder, which use a multi-level convolutional neural network to extract visible light features and infrared features respectively; Environmental data is analyzed, and then filtered and processed to retain useful information.

[0010] An environment encoder maps environmental data vectors to environmental features.

[0011] Preferably, the multimodal anomaly detection model further includes: Multimodal feature fusion module: fuses features through a gating mechanism and outputs the fused features; Anomaly detection module: Inputs fused features into the decoder to generate a fault prediction map.

[0012] Preferably, the intelligent inspection platform includes a real-time monitoring and visualization module, an anomaly detection and early warning module, a fault diagnosis and maintenance suggestion module, a data analysis and optimization module, and a remote control and management module.

[0013] Preferably, the real-time monitoring and visualization module displays the operating status of each piece of primary equipment in the hydropower station in real time, shows the working status of the equipment through a dynamic interface, and overlays fault detection results. The anomaly detection and early warning module automatically triggers an early warning when the multimodal anomaly detection model detects an equipment anomaly. The alarm information is pushed to the operation and maintenance end through sound, flashing lights and mobile devices. The fault diagnosis and maintenance suggestion module uses a large language model to analyze multimodal data of equipment faults, providing detailed fault diagnosis reports, locating fault points, and offering professional repair or replacement suggestions.

[0014] The data analysis and optimization module analyzes historical inspection data based on big data analytics to identify potential hazards and failure trends in equipment, and supports continuous optimization of detection algorithms and iterative upgrades of the system. The remote control and management module provides the operation and maintenance end with management and adjustment functions for the inspection system through a remote control interface, including equipment configuration, algorithm updates, and inspection plan adjustments.

[0015] Preferably, the method of the intelligent inspection system for primary equipment of a hydropower station based on visible light-infrared includes the following steps: S1, Acquire images of the device's exterior using a visible light camera. thermal distribution images are acquired using an infrared thermal imaging camera. Environmental parameter vectors are collected through environmental sensors. , ,in This indicates data on temperature, humidity, air pressure, or light intensity. S2, for Perform adaptive contrast-limited histogram equalization (CLAHE) to generate a preprocessed image. ;right Perform adaptive denoising based on wavelet thresholding to generate preprocessed image. ; S3, environment parameter vector Perform dimensionality normalization and project it onto the feature space; S4 is extracted through a multi-level convolutional neural network of a visible light encoder. Texture features Extracted through a multi-level convolutional neural network of an infrared encoder thermal distribution characteristics ; S5 constructs a cross-attention fusion mechanism to generate environment-gated enhanced features. and ; S6, the modal features are connected through gated residuals, and the modal features are input into the decoder to generate a fault probability map. ; ; ; in, For the sigmoid function, To fuse the weight matrix, [;] denotes feature concatenation; S7, set alarm rules and configure diagnostic prompts based on a large language model.

[0016] Preferably, step S3 includes the following steps: Environmental parameter vector Normalize to the [0,1] interval by dimension, and generate : ; in , The preset mean and standard deviation are used; Will Mapping to the feature space via linear projection: ; in Let be the projection weight matrix. This is the bias vector.

[0017] Preferably, in step S4, The hierarchical structure satisfies: ; ; in This represents the number of convolutional layers in a visible light encoder. Includes convolution, batch normalization, and ReLU activation; The hierarchical structure satisfies: ; ; ; in This represents the number of convolutional layers in the infrared encoder.

[0018] Preferably, step S5 specifically includes: Generate environment query vectors: ; Calculate visible light bond pairs: ; Calculate attention weights: ; Generation environment modulation characteristics: ; in, , and For learnable weight matrix, For feature dimensions; similarly, infrared modulation features are generated. .

[0019] Preferably, step S7 includes: When the abnormal probability value in P At that time, a level three alarm was triggered. The set threshold for outliers; Construct a diagnostic prompt: T = "Equipment location [x,y] detection [p] abnormal, environmental parameter [E~], please analyze the fault", and input the prompt into the large language model to generate a diagnostic report. .

[0020] The beneficial effects of this invention are as follows: 1. This invention effectively improves the intelligence, accuracy, and efficiency of equipment inspection by fusing visible light and infrared images and combining them with a multimodal fault detection algorithm. The system can ensure comprehensive monitoring of equipment around the clock under different lighting conditions; the visible light camera can capture external damage to the equipment under normal lighting conditions, while the infrared thermal imaging camera can effectively detect temperature anomalies in low light or harsh environments, ensuring a comprehensive understanding of the equipment's operating status.

[0021] 2. The multimodal data fusion algorithm of this invention fully leverages the complementarity of visible light and infrared images, significantly improving the accuracy and reliability of fault detection. Through deep learning for image feature extraction and fusion, the system effectively reduces the probability of missed detections and false alarms, enhancing fault identification capabilities. Furthermore, the intelligent inspection platform optimizes inspection and maintenance processes and improves work efficiency through real-time monitoring, fault early warning, and remote management functions. When a fault occurs, the system can promptly issue alarms, helping maintenance personnel quickly locate the problem and take corresponding measures, thereby reducing equipment downtime and maintenance costs. It significantly improves the efficiency and accuracy of primary equipment inspection in hydropower stations, reduces the cost and risk of manual inspection, and has high practical value and promising prospects for widespread application. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a schematic flowchart of the method of the present invention; Figure 3 This is a flowchart illustrating the multimodal anomaly detection algorithm of the present invention; Figure 4 This is a schematic diagram of the interactive interface of the intelligent inspection system of the present invention; Figure 5 This is a spatial distribution diagram of traditional AHP and improved AHP in an embodiment of the present invention. Detailed Implementation

[0023] Example 1: like Figure 1 As shown, an intelligent inspection system for primary equipment in a hydropower station based on multimodal anomaly detection includes: Data acquisition platform: consists of visible light cameras, infrared thermal imaging cameras, and environmental monitoring sensors; the environmental monitoring sensors include temperature, humidity, air pressure, and light sensors; A multimodal anomaly detection model is connected to a data acquisition platform. It uses a multi-level convolutional neural network to extract deep features from visible light and infrared images and performs fusion analysis on environmental data. The intelligent inspection platform connects to a multimodal anomaly detection model, performs real-time monitoring, anomaly detection and early warning, fault diagnosis and data analysis, and provides remote control and management.

[0024] Preferably, the multimodal anomaly detection model includes: Data preprocessing module: Based on the resolution differences, noise, and illumination changes between visible light and infrared images, preprocess the data from the infrared camera and the visible light camera respectively, and output standardized images; The feature extraction encoder includes a visible light encoder and an infrared encoder, which use a multi-level convolutional neural network to extract visible light features and infrared features respectively; Environmental data is analyzed, and then filtered and processed to retain useful information.

[0025] An environment encoder maps environmental data vectors to environmental features.

[0026] Preferably, the multimodal anomaly detection model further includes: Multimodal feature fusion module: fuses features through a gating mechanism and outputs the fused features; Anomaly detection module: Inputs fused features into the decoder to generate a fault prediction map.

[0027] Preferably, the intelligent inspection platform includes a real-time monitoring and visualization module, an anomaly detection and early warning module, a fault diagnosis and maintenance suggestion module, a data analysis and optimization module, and a remote control and management module.

[0028] Preferably, the real-time monitoring and visualization module displays the operating status of each piece of primary equipment in the hydropower station in real time, shows the working status of the equipment through a dynamic interface, and overlays fault detection results. The anomaly detection and early warning module automatically triggers an early warning when the multimodal anomaly detection model detects an equipment anomaly. The alarm information is pushed to the operation and maintenance end through sound, flashing lights and mobile devices. The fault diagnosis and maintenance suggestion module uses a large language model to analyze multimodal data of equipment faults, providing detailed fault diagnosis reports, locating fault points, and offering professional repair or replacement suggestions.

[0029] The data analysis and optimization module analyzes historical inspection data based on big data analytics to identify potential hazards and failure trends in equipment, and supports continuous optimization of detection algorithms and iterative upgrades of the system. The remote control and management module provides the operation and maintenance end with management and adjustment functions for the inspection system through a remote control interface, including equipment configuration, algorithm updates, and inspection plan adjustments.

[0030] Furthermore, the system, through the collaborative operation of visible light cameras, infrared thermal imaging cameras, and environmental sensors, acquires real-time images of the equipment's appearance, thermal imaging data, and on-site environmental information. This multimodal data is transmitted to the intelligent inspection platform, forming the system's core input. Subsequently, the multimodal anomaly detection model performs in-depth processing of the collected data, optimizing image quality and extracting key features, and combining this with environmental parameters for comprehensive analysis. This allows for accurate assessment of the equipment's operating status and identification of potential hazards and anomalies. Combined with a large language model, a detailed report is generated, including information such as fault type and handling methods.

[0031] Furthermore, after receiving the analysis results, the intelligent inspection platform responds dynamically based on different situations. For detected anomalies, the system automatically triggers an early warning mechanism, quickly conveying relevant information to the user through various means such as audible alarms, flashing lights, and mobile notifications. Simultaneously, the system generates a detailed diagnostic report, clearly identifying the fault type, specific location, and providing corresponding handling suggestions. The inspection platform's visualization function simultaneously presents the equipment's operating status, analysis results, and alarm information on the interface, helping users intuitively grasp the real-time condition of the equipment.

[0032] Furthermore, in the system design, the data analysis module deeply mines historical inspection data to identify equipment operating trends and potential hazards, providing support for the optimization of subsequent inspection plans and the iterative upgrading of algorithms. Users can also adjust inspection plans, optimize system configurations, or update algorithms through the remote control module, making the system more flexible and adaptable in actual operation.

[0033] like Figure 2 As shown, preferably, the method of the intelligent inspection system for primary equipment of a hydropower station based on visible light-infrared light includes the following steps: S1, Acquire images of the device's exterior using a visible light camera. thermal distribution images are acquired using an infrared thermal imaging camera. Environmental parameter vectors are collected through environmental sensors. , ,in This indicates data on temperature, humidity, air pressure, or light intensity. S2, for Perform adaptive contrast-limited histogram equalization (CLAHE) to generate a preprocessed image. ;right Perform adaptive denoising based on wavelet thresholding to generate preprocessed image. ; S3, environment parameter vector Perform dimensionality normalization and project it onto the feature space; S4 is extracted through a multi-level convolutional neural network of a visible light encoder. Texture features Extracted through a multi-level convolutional neural network of an infrared encoder thermal distribution characteristics ; S5 constructs a cross-attention fusion mechanism to generate environment-gated enhanced features. and ; S6, the modal features are connected through gated residuals, and the modal features are input into the decoder to generate a fault probability map. ; ; ; in, For the sigmoid function, To fuse the weight matrix, [;] denotes feature concatenation; S7, set alarm rules and configure diagnostic prompts based on a large language model.

[0034] Preferably, step S3 includes the following steps: Environmental parameter vector Normalize to the [0,1] interval by dimension, and generate : ; in , The preset mean and standard deviation are used; Will Mapping to the feature space via linear projection: ; in Let be the projection weight matrix. This is the bias vector.

[0035] Preferably, in step S4, The hierarchical structure satisfies: ; ; in This represents the number of convolutional layers in a visible light encoder. Includes convolution, batch normalization, and ReLU activation; The hierarchical structure satisfies: ; ; ; in This represents the number of convolutional layers in the infrared encoder.

[0036] Preferably, step S5 specifically includes: Generate environment query vectors: ; Calculate visible light bond pairs: ; Calculate attention weights: ; Generation environment modulation characteristics: ; in, , and For learnable weight matrix, For feature dimensions; similarly, infrared modulation features are generated. .

[0037] Preferably, step S7 includes: When the abnormal probability value in P At that time, a level three alarm was triggered. The set threshold for outliers; Construct a diagnostic prompt: T = "Equipment location [x,y] detection [p] abnormal, environmental parameter [E~], please analyze the fault", and input the prompt into the large language model to generate a diagnostic report. .

[0038] Example 2: like Figure 3 As shown, the flow of the multimodal anomaly detection algorithm is as follows: First, in the data preprocessing stage, the system processes data from infrared and visible light cameras separately. Infrared and visible light images undergo noise reduction, contrast enhancement, and resolution adjustment through their respective image preprocessing modules to ensure the quality and consistency of the input data. Simultaneously, the environmental encoder extracts environmental features from information such as temperature, humidity, air pressure, and illumination collected by environmental sensors. After normalization, these features are projected into a specific feature space for subsequent fusion with image features.

[0039] In the feature encoding stage, the infrared encoder and the visible light encoder receive preprocessed image data and extract deep features through a multi-layer convolutional neural network. These features retain the unique information of the infrared and visible light images, such as temperature distribution patterns and surface textures. Simultaneously, the environmental encoder further processes the environmental sensor data, passing the extracted environmental features to subsequent modules.

[0040] Subsequently, in the feature fusion and enhancement stage, the multimodal feature fusion module receives multi-source data from infrared encoders, visible light encoders, and environmental encoders. This module utilizes a cross-attention mechanism, combined with a gating structure, to deeply fuse environmental features with image features. Specifically, the cross-attention mechanism dynamically adjusts the weights of image features by focusing on information in the environmental data that is relevant to specific faults, thereby enhancing the features of areas in the image related to equipment anomalies. For example, in high-temperature or humid environments, the system will prioritize enhancing the feature representation of areas in the image that may exhibit corrosion or overheating, thus more accurately locating potential fault points.

[0041] After feature enhancement, the fused multimodal features are passed to the feature decoder. The feature decoder decodes these deep fused features into specific prediction result maps, and classifies fault regions and fault types according to set thresholds.

[0042] Finally, the anomaly diagnosis module inputs the detected equipment anomalies, fault types, and locations into the large language model according to preset prompts, guiding it to generate a detailed fault diagnosis report for user reference.

[0043] like Figure 4 As shown, the intelligent inspection system designed in this invention features a comprehensive and clearly laid-out interactive interface that intuitively displays various information during the inspection process, facilitating real-time monitoring and operation by users. The interface employs a modular design, rationally dividing different functional areas to meet users' needs for monitoring, equipment management, and personnel scheduling. The specific structure is as follows: In the upper left corner of the interface, areas 1, 2, 3, and 4 display thumbnails of the real-time monitoring. Areas 1 and 2 correspond to the visible light and infrared images of a specific area, while areas 3 and 4 correspond to the visible light and infrared images of another area. Users can flexibly adjust the monitoring areas corresponding to these images using the monitoring screen switching bar in area 7. These thumbnails not only display the monitoring images but also visually indicate the device's operating status using colors or markers, such as red markers for anomalies, helping users quickly identify abnormalities.

[0044] Area 5 is a schematic diagram of the distribution of primary equipment in the hydropower station, using visualization technology to dynamically display the status information of the equipment. The operating status of each piece of equipment is color-coded, such as green for normal operation, yellow for warning, and red for a fault. It also supports clicking on a specific piece of equipment to view detailed information and switch to the corresponding monitoring screen.

[0045] Area 6 is the notification bar. When the inspection system detects an equipment anomaly, the anomaly information will be displayed in this area via pop-up windows or scrolling messages. The information includes the equipment name, anomaly type, time, and location. Users can directly click on the notification item to quickly jump to the relevant monitoring screen or detailed information interface.

[0046] Area 7 in the center of the interface displays a large real-time monitoring image. Users can click on thumbnails 1, 2, 3, and 4 to select the monitoring screen they want to zoom in on, allowing for detailed observation of the operational status of a specific area. This area supports multiple view switching options, such as overlay display of visible light and infrared images, split-screen display, or dynamic switching. Furthermore, users can zoom in, zoom out, and annotate the image.

[0047] Area 8 is the user control panel, providing the core functions of system operation. The control panel includes buttons for starting and pausing equipment inspection, a fault repair confirmation button, an alarm record query box, a monitoring screen switching bar, and a data export button. This area also features a visual timeline slider, allowing users to view monitoring records for a specific time period.

[0048] Section 9 is the equipment information panel, which lists all equipment according to equipment category or region. Users can click to expand and view detailed equipment information, including equipment model, operating parameters, historical fault records, etc. The equipment information panel also supports keyword search to help users quickly locate specific equipment.

[0049] Area 10 is the staff information section, displaying information about the currently on-duty or responsible personnel, including their name, contact information, and area of ​​responsibility. Users can use this module to quickly contact relevant personnel to arrange equipment inspections or maintenance. In addition, the information section also supports sending notifications to ensure personnel receive timely information about equipment malfunctions.

[0050] Example 3: This embodiment provides a case of specifically implementing the method, the process of which includes: In the intelligent inspection of high-voltage switchgear of primary equipment in a hydropower station, three technical solutions were used to conduct a comparative analysis of fault detection over 30 consecutive days: a. Existing technology - traditional visible light surveillance; b. Existing technology - infrared thermal imaging monitoring; c. This solution's multimodal fusion system; I. Test conditions are as follows: Test equipment: 10 high-voltage switchgear units; Environmental changes: including 5 days of heavy rain, 7 days of high temperatures above 35℃, and 3 days of heavy fog; Real faults: 15 typical faults were artificially implanted, including 8 overheating faults, 4 cracks, and 3 corrosion faults.

[0051] II. Performance comparison data is shown in Table 1 below: Table 1: Comparison results with two existing technologies;

[0052] From the above table 1 and Figure 5 It is evident that this solution significantly improves the inspection performance of hydropower station equipment through multimodal fusion and cross-attention mechanisms. The fault detection accuracy reaches 97.2%, an improvement of 24.9 percentage points compared to visible light monitoring and 10.5 percentage points compared to infrared monitoring. Compared to existing technologies, this method can dynamically adjust the detection threshold through environmental feature projection, maintaining a severe weather detection rate of 95.8%, overcoming the limitations of single-modal methods. The cross-attention mechanism achieves deep fusion of environmental and image features, reducing the false alarm rate to 4.3%, a 76% reduction compared to traditional infrared technology. The gated residual fusion structure optimizes the calculation path, shortening the response time to 8.4 seconds, meeting the second-level fault response requirements of power equipment. Although energy consumption increases by 29%, the overall performance improvement provides reliable technical support for preventative maintenance of hydropower stations.

[0053] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the principles and essence of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An intelligent inspection system for primary equipment in a hydropower station based on multimodal anomaly detection, characterized in that, include: Data acquisition platform: consists of visible light cameras, infrared thermal imaging cameras, and environmental monitoring sensors; The environmental monitoring sensors include temperature, humidity, air pressure, and light sensors; A multimodal anomaly detection model is connected to a data acquisition platform. It uses a multi-level convolutional neural network to extract deep features from visible light and infrared images and performs fusion analysis on environmental data. The intelligent inspection platform connects to a multimodal anomaly detection model, performs real-time monitoring, anomaly detection and early warning, fault diagnosis and data analysis, and provides remote control and management.

2. The intelligent inspection system for primary equipment of a hydropower station based on multimodal anomaly detection according to claim 1, characterized in that, The multimodal anomaly detection model includes: Data preprocessing module: Based on the resolution differences, noise, and illumination changes between visible light and infrared images, preprocess the data from the infrared camera and the visible light camera respectively, and output standardized images; The feature extraction encoder includes a visible light encoder and an infrared encoder, which use a multi-level convolutional neural network to extract visible light features and infrared features respectively; Environmental data is analyzed, and then filtered and processed to retain useful information. An environment encoder maps environmental data vectors to environmental features.

3. The intelligent inspection system for primary equipment of a hydropower station based on multimodal anomaly detection according to claim 1, characterized in that, The multimodal anomaly detection model also includes: Multimodal feature fusion module: fuses features through a gating mechanism and outputs the fused features; Anomaly detection module: Inputs fused features into the decoder to generate a fault prediction map.

4. The intelligent inspection system for primary equipment of a hydropower station based on multimodal anomaly detection according to claim 1, characterized in that, The intelligent inspection platform includes a real-time monitoring and visualization module, an anomaly detection and early warning module, a fault diagnosis and maintenance suggestion module, a data analysis and optimization module, and a remote control and management module.

5. The intelligent inspection system for primary equipment of a hydropower station based on multimodal anomaly detection according to claim 4, characterized in that, The real-time monitoring and visualization module displays the operating status of each piece of primary equipment in the hydropower station in real time, shows the working status of the equipment through a dynamic interface, and overlays fault detection results. The anomaly detection and early warning module automatically triggers an early warning when the multimodal anomaly detection model detects an equipment anomaly. The alarm information is pushed to the operation and maintenance end through sound, flashing lights and mobile devices. The fault diagnosis and maintenance suggestion module uses a large language model to analyze multimodal data of equipment faults, providing detailed fault diagnosis reports, locating fault points, and offering professional repair or replacement suggestions. The data analysis and optimization module analyzes historical inspection data based on big data analytics to identify potential hazards and failure trends in equipment, and supports continuous optimization of detection algorithms and iterative upgrades of the system. The remote control and management module provides the operation and maintenance end with management and adjustment functions for the inspection system through a remote control interface, including equipment configuration, algorithm updates, and inspection plan adjustments.

6. The method of the intelligent inspection system for primary equipment of a hydropower station based on visible light-infrared as described in any one of claims 1-5, characterized in that, Includes the following steps: S1, Acquire images of the device's exterior using a visible light camera. thermal distribution images are acquired using an infrared thermal imaging camera. Environmental parameter vectors are collected through environmental sensors. , ,in This indicates data on temperature, humidity, air pressure, or light intensity. S2, for Perform adaptive contrast-limited histogram equalization (CLAHE) to generate a preprocessed image. ;right Perform adaptive denoising based on wavelet thresholding to generate preprocessed image. ; S3, environment parameter vector Perform dimensionality normalization and project it onto the feature space; S4 is extracted through a multi-level convolutional neural network of a visible light encoder. Texture features Extracted through a multi-level convolutional neural network of an infrared encoder thermal distribution characteristics ; S5 constructs a cross-attention fusion mechanism to generate environment-gated enhanced features. and ; S6, the modal features are connected through gated residuals, and the modal features are input into the decoder to generate a fault probability map. ; ； ； in, For the sigmoid function, To fuse the weight matrix, [;] denotes feature concatenation; S7, set alarm rules and configure diagnostic prompts based on a large language model.

7. The method for an intelligent inspection system for primary equipment in a hydropower station based on multimodal anomaly detection according to claim 6, characterized in that, Step S3 includes the following steps: Environmental parameter vector Normalize to the [0,1] interval by dimension, and generate : ； in , The preset mean and standard deviation are used; Will Mapping to the feature space via linear projection: ； in Let be the projection weight matrix. This is the bias vector.

8. The method for an intelligent inspection system for primary equipment in a hydropower station based on multimodal anomaly detection according to claim 7, characterized in that, In step S4, The hierarchical structure satisfies: ； ； in This represents the number of convolutional layers in a visible light encoder. Includes convolution, batch normalization, and ReLU activation; The hierarchical structure satisfies: ； ； ； in This represents the number of convolutional layers in the infrared encoder.

9. A method for an intelligent inspection system for primary equipment in a hydropower station based on multimodal anomaly detection, as described in claim 8, characterized in that... Step S5 specifically includes: Generate environment query vectors: ； Calculate visible light bond pairs: ； Calculate attention weights: ； Generation environment modulation characteristics: ； in, , and For learnable weight matrix, For feature dimensions; similarly, infrared modulation features are generated. .

10. A method for an intelligent inspection system for primary equipment in a hydropower station based on multimodal anomaly detection, as described in claim 9, characterized in that, Step S7 includes: When the abnormal probability value in P At that time, a level three alarm was triggered. The set threshold for outliers; Construct the diagnostic prompt: T="Equipment location [x,y] detection [p] abnormal, environmental parameter [E~], please analyze the fault", and input the prompt into the large language model to generate a diagnostic report. .

Images