A temperature anomaly identification method and device for dual-optical image fusion of hydroelectric equipment

Abstract

This invention relates to the field of hydropower equipment inspection, and in particular to a method and apparatus for temperature anomaly identification through dual-light image fusion of hydropower equipment. The method includes the following steps: acquiring infrared and visible light dual-light images of the hydropower equipment; using a rigid-flexible registration method based on feature points of key components of the equipment, combined with a noise reduction algorithm specific to humid environments, to generate a preprocessed dual-light image set; based on the preprocessed dual-light image set, extracting visible light texture features and infrared temperature grayscale features according to the temperature sensitivity of different detection parts of the hydropower equipment, generating a feature vector set representing the equipment status; based on the feature vector set, dynamically adjusting the fusion weights of the two types of features in conjunction with the real-time operating conditions of the hydropower equipment, and constructing a standardized fusion feature matrix adapted to the detection requirements after weighted fusion; dividing the standardized fusion feature matrix into a training set and a test set according to a preset ratio, and using oversampling optimization training to address sample imbalance.

Description

Technical Field

[0001] This invention relates to the field of hydroelectric equipment testing, and in particular to a method and apparatus for identifying temperature anomalies in hydroelectric equipment by dual-light image fusion. Background Technology

[0002] As a core component of the energy supply system, the operating status of hydropower equipment directly affects the stability and security of power supply, and the identification of abnormal temperatures is a key aspect of equipment operation and maintenance.

[0003] Existing temperature anomaly identification technologies are mainly divided into three categories: manual inspection combined with portable temperature measuring instruments, relying on the experience of maintenance personnel to judge whether the equipment temperature is abnormal; single infrared image detection technology, which realizes temperature monitoring by capturing the infrared radiation signal of the equipment; and early dual-light image fusion schemes, which simply superimpose infrared temperature features and visible light image information for identification.

[0004] However, existing temperature anomaly detection technologies have the following problems: Manual inspection is inefficient and costly, cannot achieve real-time monitoring around the clock, and is prone to misjudgment and omission due to subjective judgment in complex working conditions. It also poses safety risks such as working at height. Single infrared detection is easily affected by the humidity and ambient light fluctuations in hydroelectric scenarios, and has poor noise filtering effect, resulting in insufficient accuracy in temperature anomaly identification. Early dual-light fusion technology lacked targeted preprocessing, resulting in low registration accuracy of dual-light images. The feature fusion weights were fixed and not associated with the real-time operating conditions of the equipment. At the same time, it did not solve the problem of imbalanced samples, resulting in weak model generalization ability and the inability to be dynamically optimized by adding new data. After long-term use, the recognition accuracy decreased, making it difficult to adapt to the complex and ever-changing operating environment and precise detection needs of hydropower equipment. Summary of the Invention

[0005] Based on this, it is necessary to provide a method and device for temperature anomaly identification of hydroelectric equipment by dual-light image fusion to address the above-mentioned technical problems. By using a noise reduction algorithm specifically for humid environments combined with a rigid-flexible registration method, the method can accurately filter out water vapor interference noise in hydroelectric scenarios and achieve pixel-level alignment of dual-light images. Compared with single infrared detection technology, it can significantly improve the clarity and consistency of preprocessed images and lay a high-quality data foundation for subsequent feature extraction.

[0006] This invention provides a method for temperature anomaly identification of hydroelectric equipment through dual-light image fusion, the method comprising: Infrared and visible light dual-light images of hydropower equipment are collected. Based on the feature points of key components of the equipment, a rigid-flexible registration method is adopted, combined with a noise reduction algorithm specifically for humid environments, to generate a pre-processed dual-light image set. Based on the preprocessed dual-light image set, visible light texture features and infrared temperature grayscale features are extracted for the temperature sensitivity of different detection parts of hydropower equipment to generate a feature vector set representing the equipment status. Based on the feature vector set, the fusion weights of the two types of features are dynamically adjusted in combination with the real-time operating conditions of hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to the detection requirements is constructed. The standardized fusion feature matrix is ​​divided into training and test sets according to a preset ratio. Oversampling is used to optimize the training to address the imbalance of samples, and the optimized temperature anomaly recognition model is obtained. Collect dual-light images of the hydroelectric equipment to be detected, process them according to the aforementioned steps to obtain the fused feature data to be detected, and input the fused feature data to be detected into the optimized temperature anomaly recognition model; Based on the online recognition results output by the optimized model, high-value samples are automatically selected and the feature vector set is updated. The updated data is then used to dynamically optimize the feature fusion weights and the temperature anomaly recognition model.

[0007] In one embodiment, the acquisition of infrared and visible light dual-light images of the hydropower equipment, based on feature points of key components of the equipment, employs a rigid-flexible registration method combined with a noise reduction algorithm specific to humid environments to generate a preprocessed dual-light image set, including: The system acquires infrared and visible light dual-light images of hydropower equipment, eliminates acquisition delay through image spatiotemporal synchronization calibration, and accurately extracts feature points of key components of the equipment. Based on the extracted feature points, a rigid-flexible registration method is used to complete pixel-level alignment of the two-light images. Then, a noise reduction algorithm specifically for humid environments is used to filter out water vapor interference noise and generate a preprocessed two-light image set.

[0008] In one embodiment, the step of extracting visible light texture features and infrared temperature grayscale features based on the preprocessed dual-light image set, targeting the temperature sensitivity of different detection parts of the hydroelectric equipment, and generating a feature vector set characterizing the equipment state includes: The preprocessed dual-light image set is divided into equipment detection areas, and priority is marked according to the temperature sensitivity of each area to form partitioned image groups; For different priority partitions, texture detail features of visible light images and temperature grayscale distribution features of infrared images are extracted respectively. The two types of features are associated and mapped to generate a feature vector set representing the device status.

[0009] In one embodiment, the step of dynamically adjusting the fusion weights of two types of features based on a feature vector set and considering the real-time operating conditions of the hydropower equipment, and constructing a standardized fusion feature matrix adapted to detection requirements after weighted fusion, includes: Collect real-time operating data of hydropower equipment, establish a correlation mapping between operating conditions and feature importance, and determine the initial fusion weights of visible light texture features and infrared temperature grayscale features; Based on the feature effectiveness evaluation results of the feature vector set, the initial fusion weights are dynamically adjusted, and the two types of features are integrated through a weighted fusion algorithm to construct a standardized fusion feature matrix that adapts to the detection requirements.

[0010] In one embodiment, the step of dividing the standardized fusion feature matrix into a training set and a test set according to a preset ratio, using oversampling to optimize training to address sample imbalance, and obtaining an optimized temperature anomaly recognition model includes: The standardized fusion feature matrix is ​​split into training and test sets according to a preset ratio. The distribution ratio of positive and negative samples in the training set is statistically analyzed to identify the type of sample imbalance. For imbalanced sample types, an oversampling algorithm is used to expand the minority class samples to maintain the consistency of sample feature distribution. The model is then trained based on the expanded training set to obtain an optimized temperature anomaly identification model.

[0011] In one embodiment, the acquisition of the dual-light image of the hydroelectric equipment to be detected, processing it according to the aforementioned steps to obtain the fused feature data to be detected, and inputting the fused feature data to be detected into the optimized temperature anomaly recognition model, includes: Acquire dual-light images of the hydroelectric equipment to be inspected, and complete spatiotemporal calibration, registration and noise reduction using the aforementioned preprocessing process to obtain the preprocessed image to be inspected. Visible light texture features and infrared temperature grayscale features are extracted from the preprocessed image to be detected. The fused feature data to be detected is obtained by weighting the data according to the determined fusion weights and then input into the optimized temperature anomaly recognition model.

[0012] In one embodiment, the automatic screening of high-value samples and updating of the feature vector set based on the online recognition results output by the optimized model, and the dynamic optimization of the feature fusion weights and temperature anomaly recognition model using the updated data, includes: Based on the online identification results of the optimized model, high-value samples are selected according to anomaly confidence and feature identification, and additional information related to equipment operating conditions is added. The labeled high-value samples are added to the original feature vector set to complete the update. The updated dataset is then used to recalibrate the feature fusion weights and dynamically optimize the temperature anomaly recognition model.

[0013] The present invention also provides a temperature anomaly identification device for hydroelectric equipment using dual-light image fusion, applied to the temperature anomaly identification method for hydroelectric equipment using dual-light image fusion described in any of the above embodiments, the device comprising: The preprocessing module is used to acquire infrared and visible light dual-light images of hydropower equipment. Based on the feature points of key components of the equipment, a rigid-flexible registration method is adopted, combined with a noise reduction algorithm specifically for humid environments, to generate a preprocessed dual-light image set. The feature extraction module is used to extract visible light texture features and infrared temperature grayscale features based on the preprocessed dual-light image set and the temperature sensitivity of different detection parts of hydropower equipment, and generate a feature vector set representing the equipment status. The feature fusion module is used to dynamically adjust the fusion weights of two types of features based on the feature vector set and combined with the real-time operating conditions of the hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to the detection requirements is constructed. The training module is used to divide the standardized fusion feature matrix into training and test sets according to a preset ratio, and to optimize the training by oversampling to address the imbalance of samples, thereby obtaining the optimized temperature anomaly recognition model. The detection module is used to acquire dual-light images of the hydroelectric equipment to be detected, process them according to the aforementioned steps to obtain the fused feature data to be detected, and input the fused feature data to be detected into the optimized temperature anomaly recognition model. The update module is used to automatically filter high-value samples and update the feature vector set based on the online recognition results output by the optimized model, and to dynamically optimize the feature fusion weights and temperature anomaly recognition model using the updated data.

[0014] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the temperature anomaly recognition method of dual-light image fusion for hydroelectric equipment as described above.

[0015] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the temperature anomaly identification method for dual-light image fusion of hydroelectric equipment as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the temperature anomaly identification method for dual-light image fusion of hydroelectric equipment as described above.

[0017] The aforementioned method and device for temperature anomaly identification of hydroelectric equipment using dual-light image fusion employs a noise reduction algorithm specific to humid environments combined with a rigid-flexible registration method to accurately filter water vapor interference noise in hydroelectric scenarios, achieving pixel-level alignment of dual-light images. Compared to single infrared detection technology, this significantly improves the clarity and consistency of preprocessed images, laying a high-quality data foundation for subsequent feature extraction. It prioritizes the temperature sensitivity of different parts of the hydroelectric equipment, extracting visible light texture features and infrared temperature grayscale features separately, avoiding the generality of traditional feature extraction techniques. This allows the feature vector set to more accurately represent the equipment status and reduces identification bias caused by differences in location. Furthermore, it dynamically adjusts the feature fusion weights based on real-time operating conditions, overcoming the limitations of fixed weights in early dual-light fusion technologies, making the standardized fusion feature matrix more closely aligned with the equipment. It is designed to withstand actual operating conditions, improving the adaptability of temperature anomaly identification under different working conditions; it employs oversampling optimization training to address sample imbalance, supplementing minority class samples while maintaining consistent feature distribution, thus solving the problem of weak generalization ability caused by sample imbalance in traditional models. This allows the model to stably output identification results under different equipment and different anomaly scenarios; by automatically selecting high-value samples to update the feature vector set and dynamically optimizing the fusion weights and identification model, it avoids the drawback of accuracy decline after long-term use of traditional technologies, ensuring that the model can continuously adapt to changes in equipment operating conditions and maintain high identification accuracy in the long term; it eliminates the need for 24 / 7 manual inspection, avoiding safety risks such as high-altitude operations, and significantly improves detection efficiency compared to manual inspection methods, reducing the probability of misjudgment and missed judgment, providing efficient and safe technical support for the operation and maintenance of hydropower equipment. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Fig. 1 Flowchart of the temperature anomaly identification method provided by the present invention; Fig. 2 A block diagram of the temperature anomaly identification device provided by the present invention; Fig. 3 An internal structural diagram of the computer device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The following is combined Figs. 1 to 3 This invention describes a method and apparatus for identifying temperature anomalies in hydroelectric equipment through dual-light image fusion.

[0022] In one embodiment, a method for identifying temperature anomalies in hydroelectric equipment through dual-light image fusion includes the following steps: Step S100: Acquire infrared and visible light dual-light images of the hydropower equipment. Based on the feature points of key components of the equipment, adopt a rigid-flexible registration method and a noise reduction algorithm specifically for humid environments to generate a preprocessed dual-light image set.

[0023] Step S200: Based on the preprocessed dual-light image set, and considering the temperature sensitivity of different detection parts of the hydroelectric equipment, extract visible light texture features and infrared temperature grayscale features to generate a feature vector set representing the equipment status.

[0024] Step S300: Based on the feature vector set, the fusion weights of the two types of features are dynamically adjusted in combination with the real-time operating conditions of the hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to the detection requirements is constructed.

[0025] Step S400: Divide the standardized fusion feature matrix into training and test sets according to a preset ratio, use oversampling to optimize training to address sample imbalance, and obtain the optimized temperature anomaly recognition model.

[0026] Step S500: Acquire dual-light images of the hydroelectric equipment to be detected, process them according to the aforementioned steps to obtain the fused feature data to be detected, and input the fused feature data to be detected into the optimized temperature anomaly recognition model.

[0027] Step S600: Based on the online recognition results output by the optimized model, high-value samples are automatically selected and the feature vector set is updated. The updated data is then used to dynamically optimize the feature fusion weights and the temperature anomaly recognition model.

[0028] The aforementioned temperature anomaly identification method for hydroelectric equipment using dual-light image fusion employs a humid environment-specific noise reduction algorithm combined with a rigid-flexible registration method to accurately filter water vapor interference noise in hydroelectric scenarios, achieving pixel-level alignment of dual-light images. Compared to single infrared detection technology, this significantly improves the clarity and consistency of preprocessed images, laying a high-quality data foundation for subsequent feature extraction. Prioritizing the temperature sensitivity of different parts of the hydroelectric equipment, visible light texture features and infrared temperature grayscale features are extracted separately, avoiding the generality of traditional feature extraction techniques. This allows the feature vector set to more accurately represent the equipment status and reduces identification bias caused by differences in location. Furthermore, by dynamically adjusting the feature fusion weights based on real-time operating conditions, the method overcomes the limitations of fixed weights in early dual-light fusion technologies, making the standardized fusion feature matrix more closely aligned with the actual equipment conditions. The system improves the adaptability of temperature anomaly identification under different operating conditions by monitoring actual operating status. Oversampling optimization training is employed to address sample imbalance, supplementing minority class samples while maintaining consistent feature distribution. This solves the problem of weak generalization ability caused by sample imbalance in traditional models, allowing the model to stably output identification results under different equipment and abnormal scenarios. By automatically selecting high-value samples to update the feature vector set and dynamically optimizing the fusion weights and identification model, the system avoids the drawback of decreased accuracy after long-term use of traditional technologies, ensuring that the model can continuously adapt to changes in equipment operating status and maintain high identification accuracy over the long term. It eliminates the need for 24 / 7 manual inspection, avoiding safety risks such as high-altitude operations. Compared to manual inspection, it significantly improves detection efficiency and reduces the probability of misjudgments and missed judgments, providing efficient and safe technical support for the operation and maintenance of hydropower equipment.

[0029] In one embodiment, the acquisition of infrared and visible light dual-light images of hydropower equipment, based on feature points of key components of the equipment, employs a rigid-flexible registration method combined with a noise reduction algorithm specific to humid environments to generate a pre-processed dual-light image set, including the following steps: Step S110: Acquire infrared and visible light dual-light images of the hydropower equipment, eliminate acquisition delay through image spatiotemporal synchronization calibration, and accurately extract feature points of key components of the equipment; Specifically, a unified timestamp is assigned to the dual-light equipment via the GPS timing module. The time difference between the two light acquisitions is pre-calibrated by "continuously acquiring 100 sets of dual-light images and calculating the average difference between the original timestamps of the infrared and visible light images". The unified timestamp after synchronization is obtained by subtracting the calibrated average value from the original timestamp of the infrared image. After calibration, the time difference is controlled within 5ms.

[0030] The two cameras were fixed on the same gimbal, and the external parameters were calibrated using a 500mm×500mm calibration plate with a checkerboard spacing of 50mm. The rotation relationship and translation distance between the cameras were obtained to eliminate installation position deviations.

[0031] An improved ORB algorithm (Oriented Rapid Rotation Brief algorithm) is adopted to optimize the threshold for the reflective characteristics of metal parts. The extracted targets include key components such as stator winding end fixing clips (circular features), rotor magnetic pole terminals (rectangular features), and busbar joint bolts (corner features); First, the adaptive segmentation threshold is determined by "the mean of the image gray level plus 1.2 times the standard deviation" (this coefficient is determined by comparing the feature point extraction accuracy of 10 different coefficients and selecting the one with the lowest false detection rate of 1.2 times). After segmenting the image, feature points are screened using Shi-Tomasi corner detection. At least 50 feature points are extracted for each key component, and the feature point repetition rate is not less than 95%.

[0032] Step S120: Based on the extracted feature points, a rigid-flexible registration method is performed to complete the pixel-level alignment of the dual-light images. Then, a noise reduction algorithm specific to humid environments is used to filter out water vapor interference noise and generate a preprocessed dual-light image set.

[0033] Specifically, based on the feature points extracted in step S110, the homography matrix is ​​solved using the RANSAC algorithm (Random Sample Consensus Algorithm). From the 50 feature points of each key component, 3-5 pairs of non-collinear matching feature points are randomly selected to achieve global pixel alignment of the two-light image. The calculation formula is as follows: in, These are the pixel coordinates of the visible light image. To correspond to the pixel coordinates of the registered infrared image, The elements of the homography matrix are obtained by solving for more than 3 pairs of matching feature points. This is the homography matrix, the core matrix used for global pixel alignment in two-light images.

[0034] To address the localized deformation caused by the curvature of the equipment surface, a cubic B-spline interpolation deformation model was adopted. By controlling the vertices to adjust the local pixel positions, the pixel alignment error of the registered dual-light image did not exceed one pixel. Mutual information was used to evaluate the registration effect, and a mutual information value of not less than 0.75 was considered a qualified registration (this threshold was set according to the image registration accuracy requirements in the DL / T 596-2021 Hydropower Equipment Testing Standard).

[0035] Combining the Gaussian characteristics of water vapor noise and the characteristics of salt-and-pepper noise, an adaptive weighted bilateral filtering and wavelet threshold denoising fusion algorithm is adopted to accurately filter water vapor interference. The calculation formula is as follows: in, These are the filtered pixel values. For a 3×3 filter window, The normalization coefficient is... Spatial distance weighting (adapted to the spatial distribution of water vapor noise, with a spatial coefficient set to 1.5; tested on 10 sets of humid environment images, this coefficient showed the best noise reduction effect). The gray-level similarity weight is an adaptive image gray-level range, with the gray-level coefficient being 0.1 times the maximum gray-level value of the image, determined by comparing the noise suppression rate in the 0.05-0.2 times range.

[0036] Wavelet thresholding denoising employs a 3-layer decomposition using the db4 wavelet (Daubechies 4 wavelet) (based on the texture complexity of the infrared image of hydropower equipment, the 3-layer decomposition can balance noise filtering and detail preservation). Soft thresholding is applied to high-frequency coefficients (residual noise is filtered by the rule of "retaining the difference between the absolute value of the coefficient and the threshold, and setting it to 0 if it is less than the threshold").

[0037] The peak signal-to-noise ratio of the denoised image is no less than 35dB, and the structural similarity is no less than 0.92 (this threshold refers to GB / T33190-2016 Machine Vision Image Quality Standard). Compared with traditional Gaussian filtering, the water vapor noise suppression rate is significantly improved. In one embodiment, the step of extracting visible light texture features and infrared temperature grayscale features based on a preprocessed dual-light image set, and generating a feature vector set characterizing the equipment status by considering the temperature sensitivity of different detection parts of the hydroelectric equipment, includes the following steps: Step S210: Divide the preprocessed dual-light image set into equipment detection areas, and mark the priority according to the temperature sensitivity of each area to form partitioned image groups; Specifically, the equipment inspection area division results are shown in the following example:

[0038] Sensitivity assessment metrics include normal operating temperature, abnormal temperature threshold, temperature rise rate threshold, and severity of fault consequences.

[0039] The priority criteria and results are as follows:

[0040] The preprocessed dual-light images are automatically cropped by part, the cropping window is located based on the coordinates of feature points, the edge redundancy does not exceed 10 pixels, and three types of partitioned image groups, P1, P2 and P3, are generated. Each group contains no less than 1,000 pairs of infrared and visible light images.

[0041] Step S220: For different priority partitions, extract the texture detail features of the visible light image and the temperature grayscale distribution features of the infrared image respectively, associate and map the two types of features to generate a feature vector set representing the device status.

[0042] Specifically, fine-grained texture features (LBP, GLCM, Gabor filter responses) are extracted from P1 / P2 level regions, while coarse-grained texture features are extracted from P3 level regions.

[0043] LBP (Local Binary Pattern): Using a circular neighborhood with a radius of 3 or 8 pixels, the neighboring pixels are compared with the center pixel. If the neighboring pixels are greater than or equal to the center pixel, they are recorded as 1; otherwise, they are recorded as 0. These are used to form a 256-dimensional histogram feature. GLCM (Gray Co-occurrence Matrix): Take four angles at distances of 1, 0° / 45° / 90° / 135°, calculate four statistical measures: contrast, correlation, energy, and entropy, and generate 16-dimensional features; Gabor filtering: It uses 5 scales and 8 directions to generate 40-dimensional filter response features (the number of scales and directions is determined by cross-validation, which can cover common texture types of hydropower equipment components).

[0044] Among them, the texture feature dimension of P1 / P2 level parts is 312 dimensions, and that of P3 level parts is 272 dimensions.

[0045] The original temperature measurement values ​​of the infrared image are normalized to a gray value of 0-255. The normalization range is set according to the priority of the body part (P1 level 40-120℃, P2 level 30-150℃, P3 level 20-200℃). Extract the global grayscale histogram (256-dimensional), mean, variance, peak position (4-dimensional), and grayscale mean and gradient maximum (2-dimensional) of the temperature anomaly candidate region. All priority parts are unified as 262-dimensional features.

[0046] Using "feature alignment and weighted concatenation", texture features are mapped one-to-one with temperature features based on location coordinates. The calculation formula is as follows: in, For the final feature vector, For texture feature vectors, This is the temperature grayscale feature vector. This is a vector concatenation operation. The association weights are set as follows (0.4 for P1 / P2 level, 0.3 for P3 level, and the weight values ​​with the highest model recognition accuracy are selected through 5-fold cross-validation).

[0047] The operation sequence is to first weight the two types of feature vectors separately, and then perform the concatenation operation.

[0048] Sort all feature vectors of different body parts according to their priority P1→P2→P3 to construct a feature vector set. The dimension is determined by the combination of the number of samples and the priority of the body parts.

[0049] In one embodiment, the step of constructing a standardized fusion feature matrix adapted to detection requirements by dynamically adjusting the fusion weights of two types of features based on the feature vector set and the real-time operating conditions of the hydropower equipment, and then performing weighted fusion, includes the following steps: Step S310: Collect real-time operating condition data of hydropower equipment, establish a correlation mapping between operating condition and feature importance, and determine the initial fusion weights of visible light texture features and infrared temperature grayscale features. Specifically, the collected parameters and sources include unit load rate (generator PLC control system, 1Hz acquisition), rotor speed (speed sensor, 10Hz acquisition), ambient humidity (plant temperature and humidity sensor, 0.5Hz acquisition), cooling water flow rate (cooling system flow meter, 1Hz acquisition), and runtime (equipment operation log, updated every 1 minute). A moving average filter with 10 data points is used to eliminate noise, and missing values ​​are supplemented by linear interpolation. The data and image timestamp error does not exceed 1 second.

[0050] A random forest feature importance assessment model is constructed, with operating parameters as input and feature information gain as output, to evaluate the importance of texture features and temperature features (measured by the reduction of information entropy after feature partitioning).

[0051] The mapping relationship is represented as follows:

[0052] Based on the importance ratio of the two types of features, the initial weights of visible light texture features and infrared temperature grayscale features are determined to ensure that the sum of the weights is 1. The weight of texture features is controlled between 0.3 and 0.6 to avoid a single feature dominating, and the boundary value is taken when it exceeds the range. When the working conditions are 85% load rate, 720 r / min speed, 90% humidity and 1.5 m / s water flow rate, the initial weight of texture features is 0.35 and the initial weight of temperature features is 0.65.

[0053] Step S320: Based on the feature effectiveness evaluation results of the feature vector set, dynamically adjust the initial fusion weights, integrate the two types of features through a weighted fusion algorithm, and construct a standardized fusion feature matrix that adapts to the detection requirements.

[0054] Specifically, variance contribution rate (measures the ability of a feature to distinguish between normal and abnormal states, with a value range of 0-1, the larger the value, the more effective the feature) and inter-class distance (the Euclidean distance between the feature sets of normal and abnormal samples, the larger the value, the more effective the feature) are used. Each batch consists of 100 samples. The variance contribution rate and inter-class distance of the two types of features are calculated, and the average value is taken as the feature effectiveness score for that batch.

[0055] The initial weights are dynamically adjusted based on the ratio of the effectiveness scores of the two types of features. At the same time, an exponential moving average is used (the current adjusted weight is multiplied by 0.7, plus the final weight of the previous batch multiplied by 0.3) to avoid sudden changes in weights and ensure a smooth adjustment process. The initial weights are 0.35 for texture features and 0.65 for temperature features. In a certain batch, the effectiveness scores of texture features are 0.72 and temperature features are 0.88. After adjustment, the weight of texture features is 0.3225 and the weight of temperature features is 0.6775.

[0056] Based on the adjusted final weights, the texture and temperature features of each sample are weighted and summed to obtain the fused feature vector of a single sample. Z-score standardization is used to eliminate the influence of dimensions (the fused feature vector is subtracted from the feature mean of the training set and then divided by the feature standard deviation of the training set). All standardized sample fused feature vectors are arranged in rows to construct a standardized fused feature matrix with the matrix dimension being the total number of samples × the fused feature dimension of a single sample.

[0057] In one embodiment, the step of dividing the standardized fusion feature matrix into a training set and a test set according to a preset ratio, using oversampling to optimize training to address sample imbalance, and obtaining an optimized temperature anomaly recognition model includes the following steps: Step S410: Split the standardized fusion feature matrix into a training set and a test set according to a preset ratio, count the distribution ratio of positive and negative samples in the training set, and identify the type of sample imbalance. Specifically, stratified sampling is performed at an 8:2 ratio to ensure that the ratio of normal / abnormal samples in the training set and the test set is consistent, thus avoiding sampling bias. Normal samples are labeled as 0, and abnormal samples are labeled as 1 (according to DL / T 596-2021: exceeding the temperature threshold or the heating rate threshold is considered abnormal).

[0058] Calculate the ratio of normal samples to abnormal samples in the training set. An example of the type classification criteria is as follows:

[0059] For example, a training set contains 40,000 normal samples and 320 abnormal samples, with a sample ratio of 125 times, which is judged as extremely imbalanced.

[0060] Step S420: For imbalanced sample types, an oversampling algorithm is used to expand the minority class samples to maintain the consistency of sample feature distribution. The model is trained based on the expanded training set to obtain the optimized temperature anomaly identification model.

[0061] Specifically, an improved ADASYN algorithm (adaptive synthetic sampling algorithm) is adopted, optimized for the sparse distribution of outlier samples. For each outlier sample, one of its five nearest neighbors is randomly selected (the number of neighbors is determined by comparing the generation quality of samples from 3-7 nearest neighbors), and a new sample is generated with a random coefficient between 0 and 1. The coefficient range for marginal samples is limited to 0.3-0.7 to avoid deviation from the true distribution. The expansion quantity is determined to be 30 times the target sample ratio (the target ratio is referenced to the commonly used balance threshold in the field of imbalanced learning). In this example, 30,400 outlier samples need to be synthesized, and after expansion, the total number of outlier samples is 30,720, with a sample ratio of approximately 1.3 times (close to balance).

[0062] The KS test was used to verify the consistency of the feature distribution between the synthetic sample and the original outlier sample (by comparing the differences in the cumulative feature distribution curves of the two types of samples, the maximum difference value should be less than 0.05); the feature outlier factor of the synthetic sample was calculated (judged by the density ratio of the sample to the neighboring samples), and samples with a value greater than 1.5 were regarded as outliers and removed to avoid introducing noise.

[0063] The lightweight CNN model MobileNetV3-Small was selected to adapt to the deployment of industrial edge devices. The optimizer was AdamW (learning rate 1e-4, weight decay 1e-5), and the loss function was Focal Loss (focusing on key samples by "reducing the weight of easily classified samples and increasing the weight of difficult-to-classify samples", with the focus coefficient set to 2, and the optimal value for recall was selected by comparing coefficients between 1 and 3). The training was conducted for 50 epochs, and an early stopping strategy was adopted (the training was stopped if the accuracy on the validation set did not improve for 5 consecutive epochs). On the test set, the accuracy was no less than 98%, the recall rate of abnormal samples was no less than 95%, the precision was no less than 90%, and the F1 score was no less than 92% (the F1 score was calculated by the harmonic mean of precision and recall, and this performance index met the real-time and accuracy requirements of online monitoring of hydropower equipment).

[0064] In one embodiment, the process of acquiring the dual-light image of the hydroelectric equipment to be detected, processing it according to the aforementioned steps to obtain the fused feature data to be detected, and inputting the fused feature data to be detected into the optimized temperature anomaly identification model includes the following steps: Step S510: Acquire dual-light images of the hydroelectric equipment to be inspected, and complete spatiotemporal calibration, registration and noise reduction processing using the aforementioned preprocessing process to obtain the preprocessed image to be inspected; Specifically, the acquisition parameters (frame rate, exposure time, temperature measurement range) remain fixed; manual triggering or timed triggering (5-minute interval) is supported, and the current operating parameters of the device are recorded synchronously during acquisition and bound to the image timestamp.

[0065] The GPS timing synchronization scheme is used, and the timestamp error is controlled within 5ms. The dual-light camera extrinsic parameters and homography matrix pre-stored during the training phase are directly called. When the equipment is stationary, there is no need for recalibration. When the equipment is moved, it is automatically detected and recalibrated through the calibration board. The registered image is Gaussian blurred (coefficient 0.5) to eliminate residual jagged edges from the registration.

[0066] The parameters of the fusion and denoising algorithm used in the training phase are used to ensure consistent processing standards. The peak signal-to-noise ratio of the preprocessed image to be detected is not less than 33dB and the structural similarity is not less than 0.90 (this threshold is lower than the training set standard and is adapted to the image quality fluctuations in real-time on-site acquisition). Images that do not meet the standards are re-acquired (up to 3 times). If the standards are still not met after 3 times, an alarm is triggered, prompting maintenance personnel to check the acquisition environment or equipment status.

[0067] Step S520: Extract visible light texture features and infrared temperature grayscale features from the preprocessed image to be detected, weight them according to the determined fusion weights to obtain the fusion feature data to be detected, and input them into the optimized temperature anomaly recognition model.

[0068] Specifically, the feature extraction methods from the training phase are fully reused, and the parameters of LBP, GLCM, and Gabor filtering, as well as the threshold ranges for temperature and grayscale mapping, remain unchanged. The size of the texture feature extraction window, neighborhood parameters, distance and angle of GLCM are fixed, and the temperature feature normalization adopts the temperature range from the training phase, which does not change with the temperature of the image to be detected.

[0069] The system collects the operating parameters corresponding to the image to be detected in real time, inputs the operating conditions, feature importance association mapping model and dynamic adjustment algorithm to obtain the final fusion weight under the current operating conditions; the texture features and temperature features are weighted and summed according to the final weight to obtain the fusion feature data to be detected; the system uses the feature mean and standard deviation pre-stored in the training stage for standardization to avoid feature shift caused by statistical bias of the sample to be detected.

[0070] The standardized fusion feature vector to be detected is adjusted to a single sample dimension; the inference precision is set to FP16 (balancing speed and precision; 16-bit floating point), and the inference time does not exceed 100ms (to meet the real-time detection requirements); the model outputs anomaly confidence (between 0 and 1), and a confidence score not lower than 0.85 is judged as a temperature anomaly (this threshold is calibrated by the false alarm rate of the test set to ensure that the false alarm rate is less than 1%), and a confidence score lower than 0.85 is judged as normal.

[0071] In one embodiment, the process of automatically selecting high-value samples and updating the feature vector set based on the online recognition results output by the optimized model, and dynamically optimizing the feature fusion weights and temperature anomaly recognition model using the updated data, includes the following steps: Step S610: Based on the online identification results of the optimized model, high-value samples are selected according to anomaly confidence and feature identification, and additional information related to equipment operating conditions is added. Specifically, high-value positive samples (abnormal) have a confidence level of 0.85-0.95, and high-value negative samples (normal) have a confidence level of 0.05-0.15. Feature entropy is used as the measure (calculated by multiplying the values ​​of each feature dimension after min-max normalization by the sum of their negative logarithms). An entropy value of not less than 10 is considered to be rich in feature information and highly identifiable (this threshold is determined by statistically analyzing the entropy distribution of normal / abnormal samples). Candidate samples are first screened based on confidence level, and then feature entropy is calculated. If both conditions are met, the sample is judged to be a high-value sample. The screening ratio does not exceed 5% of the total number of samples (to avoid redundancy).

[0072] The complete operating parameters (load rate, speed, humidity, water flow rate, running time), equipment operation mode (start-stop / steady-state / peak shaving), and environmental interference factors (rainfall / condensation) corresponding to the image to be detected are automatically read and associated from the PLC system, and the environmental interference factors are supplemented and labeled by the operation and maintenance personnel using the LabelStudio tool; the labeling format is sample file name, timestamp, JSON string of operating parameters, and label (high-value positive / negative sample).

[0073] Step S620: Add the labeled high-value samples to the original feature vector set to complete the update, recalibrate the feature fusion weights using the updated dataset, and dynamically optimize the temperature anomaly recognition model.

[0074] Specifically, the update trigger condition is either "every 7 days" or "accumulated high-value samples ≥ 1000". If there are fewer than 1000 samples within 7 days, the update will still be forced every 7 days. The labeled high-value samples will be added to the original feature vector set, while low-value samples with feature entropy values ​​below 5 will be removed from the original set to keep the total number of samples stable (not exceeding 100,000). The information gain rate of the updated feature vector set will be calculated (measured by the "average increase in feature importance before and after the update"). An increase of at least 5% compared to before the update is considered a valid update.

[0075] Based on the updated feature vector set and the newly added working condition data, the working condition and feature importance correlation model is retrained, and new initial weights and dynamically adjusted final weights are calculated. The absolute difference between the new weights and the original weights does not exceed 0.1 to avoid sudden changes in weights that could cause fluctuations in model performance.

[0076] A model fine-tuning approach was adopted, freezing the first 80% of the backbone network (divided according to the proportion of convolutional layers in MobileNetV3-Small, with the first 80% being basic feature extraction layers and the last 20% being task-related layers), and training only the classification head and the last 20% of layers; the learning rate was set to 1e-5 (1 / 10 of the initial training), with 20 training epochs, and the loss function remained Focal Loss; the new model must have an anomaly recall rate of no less than 96% on the updated test set, and an improvement of no less than 1% compared to the original model, otherwise it would be rolled back to the original model; the optimized model was deployed to the edge detection device via OTA, and the deployment process did not interrupt real-time detection (seamless switching).

[0077] The temperature anomaly identification device for hydroelectric equipment by dual-light image fusion provided by the present invention is described below. The temperature anomaly identification device for hydroelectric equipment by dual-light image fusion described below can be referred to in correspondence with the temperature anomaly identification method for hydroelectric equipment by dual-light image fusion described above.

[0078] In one embodiment, a temperature anomaly identification device for hydropower equipment using dual-light image fusion is applied to the temperature anomaly identification method for hydropower equipment using dual-light image fusion described in any of the above embodiments. The device includes a preprocessing module, a feature extraction module, a feature fusion module, a training module, a detection module, and an update module.

[0079] The preprocessing module is used to acquire infrared and visible light dual-light images of hydroelectric equipment. Based on the feature points of key components of the equipment, a rigid-flexible registration method is adopted, combined with a noise reduction algorithm specifically for humid environments, to generate a preprocessed dual-light image set.

[0080] The feature extraction module is used to extract visible light texture features and infrared temperature grayscale features based on the preprocessed dual-light image set and the temperature sensitivity of different detection parts of hydroelectric equipment, and generate a feature vector set representing the equipment status.

[0081] The feature fusion module is used to dynamically adjust the fusion weights of two types of features based on the feature vector set and combined with the real-time operating conditions of the hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to the detection requirements is constructed.

[0082] The training module is used to divide the standardized fusion feature matrix into training and test sets according to a preset ratio, and to optimize the training by oversampling to address the imbalance of samples, thereby obtaining an optimized temperature anomaly recognition model.

[0083] The detection module is used to acquire dual-light images of the hydroelectric equipment to be detected, process them according to the aforementioned steps to obtain the fused feature data to be detected, and input the fused feature data to be detected into the optimized temperature anomaly recognition model.

[0084] The update module is used to automatically filter high-value samples and update the feature vector set based on the online recognition results output by the optimized model, and to dynamically optimize the feature fusion weights and temperature anomaly recognition model using the updated data.

[0085] Fig. 3 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Fig. 3 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for temperature anomaly identification through dual-light image fusion in hydroelectric equipment.

[0086] Those skilled in the art will understand that Fig. 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device to which the present invention is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0087] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, realizes a method for temperature anomaly identification by dual-light image fusion of hydroelectric equipment.

[0088] On another front, a computer program product or computer program is provided, comprising computer instructions stored in a computer storage medium. A processor of an electronic device reads the computer instructions from the computer storage medium, and when the processor executes the computer instructions, it implements a method for temperature anomaly identification through dual-light image fusion of hydroelectric equipment.

[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0090] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for temperature anomaly identification through dual-light image fusion of hydroelectric equipment, characterized in that, The method includes: Infrared and visible light dual-light images of hydropower equipment are collected. Based on the feature points of key components of the equipment, a rigid-flexible registration method is adopted, combined with a noise reduction algorithm specifically for humid environments, to generate a pre-processed dual-light image set. Based on the preprocessed dual-light image set, visible light texture features and infrared temperature grayscale features are extracted for the temperature sensitivity of different detection parts of hydropower equipment to generate a feature vector set representing the equipment status. Based on the feature vector set, the fusion weights of the two types of features are dynamically adjusted in combination with the real-time operating conditions of hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to the detection requirements is constructed. The standardized fusion feature matrix is ​​divided into training and test sets according to a preset ratio. Oversampling is used to optimize the training to address the imbalance of samples, and the optimized temperature anomaly recognition model is obtained. Collect dual-light images of the hydroelectric equipment to be detected, process them according to the aforementioned steps to obtain the fused feature data to be detected, and input the fused feature data to be detected into the optimized temperature anomaly recognition model; Based on the online recognition results output by the optimized model, high-value samples are automatically selected and the feature vector set is updated. The updated data is then used to dynamically optimize the feature fusion weights and the temperature anomaly recognition model.

2. The method for temperature anomaly identification by dual-light image fusion of hydroelectric equipment according to claim 1, characterized in that, The acquired infrared and visible light dual-light images of the hydropower equipment are processed using a rigid-flexible registration method based on feature points of key components of the equipment, combined with a noise reduction algorithm specific to humid environments, to generate a preprocessed dual-light image set, including: The system acquires infrared and visible light dual-light images of hydropower equipment, eliminates acquisition delay through image spatiotemporal synchronization calibration, and accurately extracts feature points of key components of the equipment. Based on the extracted feature points, a rigid-flexible registration method is used to complete pixel-level alignment of the two-light images. Then, a noise reduction algorithm specifically for humid environments is used to filter out water vapor interference noise and generate a preprocessed two-light image set.

3. The method for temperature anomaly identification of hydroelectric equipment by dual-light image fusion according to claim 2, characterized in that, The preprocessed dual-light image set, based on the temperature sensitivity of different detection parts of the hydroelectric equipment, extracts visible light texture features and infrared temperature grayscale features to generate a feature vector set characterizing the equipment status, including: The preprocessed dual-light image set is divided into equipment detection areas, and priority is marked according to the temperature sensitivity of each area to form partitioned image groups; For different priority partitions, texture detail features of visible light images and temperature grayscale distribution features of infrared images are extracted respectively. The two types of features are associated and mapped to generate a feature vector set representing the device status.

4. The method for temperature anomaly identification of hydroelectric equipment by dual-light image fusion according to claim 3, characterized in that, The method involves dynamically adjusting the fusion weights of two types of features based on a feature vector set and considering the real-time operating conditions of the hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to detection requirements is constructed, including: Collect real-time operating data of hydropower equipment, establish a correlation mapping between operating conditions and feature importance, and determine the initial fusion weights of visible light texture features and infrared temperature grayscale features; Based on the feature effectiveness evaluation results of the feature vector set, the initial fusion weights are dynamically adjusted, and the two types of features are integrated through a weighted fusion algorithm to construct a standardized fusion feature matrix that adapts to the detection requirements.

5. The method for temperature anomaly identification by dual-light image fusion of hydroelectric equipment according to claim 4, characterized in that, The process of dividing the standardized fusion feature matrix into training and testing sets according to a preset ratio, optimizing the training by oversampling to address sample imbalance, and obtaining an optimized temperature anomaly recognition model includes: The standardized fusion feature matrix is ​​split into training and test sets according to a preset ratio. The distribution ratio of positive and negative samples in the training set is statistically analyzed to identify the type of sample imbalance. For imbalanced sample types, an oversampling algorithm is used to expand the minority class samples to maintain the consistency of sample feature distribution. The model is then trained based on the expanded training set to obtain an optimized temperature anomaly identification model.

6. The method for temperature anomaly identification of hydroelectric equipment by dual-light image fusion according to claim 5, characterized in that, The acquired dual-light image of the hydroelectric equipment to be detected is processed according to the aforementioned steps to obtain the fused feature data to be detected, and the fused feature data to be detected is input into the optimized temperature anomaly recognition model, including: Acquire dual-light images of the hydroelectric equipment to be inspected, and complete spatiotemporal calibration, registration and noise reduction using the aforementioned preprocessing process to obtain the preprocessed image to be inspected. Visible light texture features and infrared temperature grayscale features are extracted from the preprocessed image to be detected. The fused feature data to be detected is obtained by weighting the data according to the determined fusion weights and then input into the optimized temperature anomaly recognition model.

7. The method for temperature anomaly identification by dual-light image fusion of hydroelectric equipment according to claim 6, characterized in that, The online identification results based on the optimized model output automatically filter high-value samples and update the feature vector set. The updated data is then used to dynamically optimize the feature fusion weights and temperature anomaly identification model, including: Based on the online identification results of the optimized model, high-value samples are selected according to anomaly confidence and feature identification, and additional information related to equipment operating conditions is added. The labeled high-value samples are added to the original feature vector set to complete the update. The updated dataset is then used to recalibrate the feature fusion weights and dynamically optimize the temperature anomaly recognition model.

8. A temperature anomaly identification device for hydroelectric equipment using dual-light image fusion, applied to the temperature anomaly identification method for hydroelectric equipment using dual-light image fusion as described in any one of claims 1 to 7, characterized in that, The device includes: The preprocessing module is used to acquire infrared and visible light dual-light images of hydropower equipment. Based on the feature points of key components of the equipment, a rigid-flexible registration method is adopted, combined with a noise reduction algorithm specifically for humid environments, to generate a preprocessed dual-light image set. The feature extraction module is used to extract visible light texture features and infrared temperature grayscale features based on the preprocessed dual-light image set and the temperature sensitivity of different detection parts of hydropower equipment, and generate a feature vector set representing the equipment status. The feature fusion module is used to dynamically adjust the fusion weights of two types of features based on the feature vector set and combined with the real-time operating conditions of the hydropower equipment. After weighted fusion, a standardized fusion feature matrix adapted to the detection requirements is constructed. The training module is used to divide the standardized fusion feature matrix into training and test sets according to a preset ratio, and to optimize the training by oversampling to address the imbalance of samples, thereby obtaining the optimized temperature anomaly recognition model. The detection module is used to acquire dual-light images of the hydroelectric equipment to be detected, process them according to the aforementioned steps to obtain the fused feature data to be detected, and input the fused feature data to be detected into the optimized temperature anomaly recognition model. The update module is used to automatically filter high-value samples and update the feature vector set based on the online recognition results output by the optimized model, and to dynamically optimize the feature fusion weights and temperature anomaly recognition model using the updated data.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images