Self-adaptive temperature anomaly detection method, system, equipment and medium

Abstract

The invention discloses a self-adaptive temperature anomaly detection method, system and device and a medium, and the method comprises the steps: carrying out the feature extraction of preprocessed first modal data and second modal data through a specific encoder, and obtaining a first feature representation and a second feature representation of the first modal data and the second modal data; fusing the first feature representation and the second feature representation through an adaptive fusion mechanism to obtain a fused modal feature; effective historical frame features are screened through a memory dynamic screening mechanism, and dynamic memory features of the first feature representation, the second feature representation and the fused modal features are obtained; performing multi-modal memory matching on the current feature and the dynamic memory feature to obtain a matched fusion feature; and converting the matched fusion features through a decoder to obtain a specific position containing the temperature anomaly of the power equipment. On the premise of ensuring the detection accuracy, the detection performance of the model under various environmental conditions is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power system monitoring and fault diagnosis, and particularly to an adaptive temperature anomaly detection method, system, device and medium. Background Art

[0002] The accuracy of temperature anomaly detection in power systems is crucial for the safe operation of power equipment. Due to the complex and variable environment where power equipment is located, a single sensor often fails to meet the monitoring requirements for all-weather and full-scenario conditions. Especially under complex weather and lighting conditions, the imaging quality of visible light cameras will be severely affected. The combination of multi-modal sensors, such as the visible light-depth-infrared (RGB-D-T) scheme, provides natural advantages for achieving stable and reliable temperature anomaly detection.

[0003] Deep learning models provide a general method for coordinating the processing of visible light, depth, and infrared information by fusing multi-modal data, which is crucial for accurately detecting temperature anomalies in power equipment. However, the data characteristics of different modal sensors often vary: visible light images provide rich texture details, infrared images reflect temperature distributions, and depth images contain spatial geometric information. Since these characteristics are each unique, in order to ensure the accuracy of detection, it is necessary to reasonably fuse multi-modal data, achieve complementarity at the feature level, and improve the robustness of the detection model.

[0004] The training and inference of deep learning models often consume a large amount of computing resources due to their large computational volume. Especially when processing multi-modal data, simple feature concatenation or average fusion methods not only fail to fully utilize the advantages of different modal data but also result in a waste of computing resources. This resource waste not only reduces the computational efficiency of the detection model but also limits the deployment and application of the model in actual scenarios, affecting the timely monitoring of temperature anomalies in power equipment. Summary of the Invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides an adaptive temperature anomaly detection method, system, device and medium to solve the significant defects existing in the existing multi-modal temperature detection technology in complex environments, especially the problem of unstable detection performance under conditions such as low light, rain and fog weather, and complex backgrounds.

[0007] To solve the above technical problems, the present invention provides the following technical solutions:

[0008] In the first aspect, the present invention provides an adaptive temperature anomaly detection method, including:

[0009] Obtain first-modal data and second-modal data;

[0010] For the preprocessed first-modal data and second-modal data, specific encoders are used to extract features respectively, obtaining the respective first feature representations and second feature representations;

[0011] The first feature representation and the second feature representation are fused through an adaptive fusion mechanism to obtain fused-modal features;

[0012] Based on the fused-modal features, effective historical frame features are screened through a memory dynamic screening mechanism to obtain the dynamic memory features of the first feature representation, the second feature representation, and the fused-modal features;

[0013] The current feature is subjected to multi-modal memory matching with the dynamic memory features to obtain the fused features after matching;

[0014] The fused features after matching are transformed through a decoder to obtain the specific location containing the temperature anomaly of the power equipment.

[0015] As a preferred solution of the adaptive temperature anomaly detection method described in the present invention, wherein:

[0016] The preprocessing includes the following steps:

[0017] The first feature point matching algorithm is adopted to perform spatial alignment on the first-modal data and the second-modal data;

[0018] Cropping is performed according to the specific location and area of the power equipment;

[0019] The first-modal data and the second-modal data are normalized;

[0020] The temperature anomaly regions of the second-modal data are labeled using historical data.

[0021] As a preferred solution of the adaptive temperature anomaly detection method described in the present invention, wherein:

[0022] The adaptive fusion mechanism includes the following steps:

[0023] According to the current environmental parameters, the weight coefficients of the first-modal data and the second-modal data are dynamically calculated;

[0024] The attention matrices of the first-modal data and the second-modal data are calculated;

[0025] The first feature fusion strategy is adopted to combine the obtained weight coefficients and attention matrices to obtain the first comprehensive feature.

[0026] The beneficial effects of this preferred technical solution are as follows. By effectively integrating multi-modal information, the overall performance of the system is enhanced, including but not limited to improving feature expression, increasing robustness and generalization ability, enhancing information utilization efficiency, and the ability to adapt to environmental changes.

[0027] As a preferred solution of the adaptive temperature anomaly detection method described in the present invention, wherein:

[0028] The memory dynamic screening mechanism includes the following steps:

[0029] By establishing a feature memory bank, calculating the contribution degree of each modal historical frame to the accuracy of calculating temperature anomaly points, and obtaining the contribution degree evaluation result of each frame of feature;

[0030] According to the contribution degree evaluation result of the current frame feature, set a dynamic screening threshold, dynamically update the memory bank, and select relevant historical frame features;

[0031] Adopt the first feature matching algorithm to integrate the historical frame features into the current frame features to obtain the dynamic memory features of the first feature representation, the second feature representation, and the fused modal features.

[0032] The beneficial effects of this preferred technical solution are as follows. By effectively managing historical frames, not only can the calculation efficiency and accuracy be improved, but also the adaptability and anti-noise ability of the system can be enhanced.

[0033] As a preferred solution of the adaptive temperature anomaly detection method described in the present invention, wherein:

[0034] The multi-modal memory matching of the current feature and the dynamic memory feature includes the following steps:

[0035] Through an adaptive matching strategy, match the current frame feature with the dynamic memory features of the first feature representation, the second feature representation, and the fused modal features;

[0036] Adopt an attention mechanism to dynamically adjust the importance weights of each feature to obtain the fused feature after matching.

[0037] As a preferred solution of the adaptive temperature anomaly detection method described in the present invention, wherein:

[0038] The dynamic calculation of the weight coefficients of the first modal data and the second modal data is expressed as:

[0039]

[0040] W T (t) = σ(f env (T(t), W(t), C(t)))

[0041] Wherein, WRGB (t) and W T (t) represent the weight coefficients of the RGB and thermal infrared modalities at time t, L(t) represents the current light intensity, W(t) represents the weather condition parameter, C(t) represents the scene complexity, T(t) represents

[0042] the ambient temperature, f env is the environmental assessment function, and σ is the normalization function

[0043] The calculation of the attention matrix for the first modal data and the second modal data is expressed as:

[0044]

[0045] where, A RGB and A T are the attention matrices of the first modal data and the second modal data respectively, W q and W k are learnable query and key-value parameter matrices, is the square root of the feature dimension.

[0046] The first comprehensive feature F F is expressed as:

[0047]

[0048] where, represents the concatenation operation of the feature channel dimensions, α(t) is the adaptive fusion coefficient, and Conv is the convolution operation.

[0049] As a preferred solution of the adaptive temperature anomaly detection method described in the present invention, wherein:

[0050] The calculation of the contribution degree of each modal historical frame to the accuracy of calculating temperature anomaly points is expressed as:

[0051] C(F i ) = exp(-·MSE(Pred(F i ), GT))·(1 - exp(-γ·Age(F i )))

[0052] where, MSE is the mean square error between the prediction result and the true label, Pred(F i ) is the prediction result using the feature F i , GT is the true label, Age(F i ) is the time decay factor of the feature, and λ and γ are adjustable hyperparameters;

[0053] The dynamic update of the memory bank is expressed as:

[0054] M t = {F j | C(F j ) > θ t , j ∈ [t - k, t]}θ t = μ · mean({C(F j ) | j ∈ [t - k, t]})

[0055] Among them, M t represents the memory feature library at time t, θ t is the dynamic screening threshold, k is the number of historical frames considered, and μ is the threshold adjustment coefficient.

[0056] In the second aspect, the present invention provides an adaptive temperature anomaly detection system, including:

[0057] A data acquisition module for acquiring first-modal data and second-modal data;

[0058] A feature extraction module for respectively using a specific encoder to extract features from the preprocessed first-modal data and second-modal data to obtain respective first feature representations and second feature representations;

[0059] An adaptive fusion module for fusing the first feature representation and the second feature representation through an adaptive fusion mechanism to obtain a fused modal feature;

[0060] A memory dynamic screening module for screening effective historical frame features based on the fused modal feature through a memory dynamic screening mechanism to obtain dynamic memory features of the first feature representation, the second feature representation, and the fused modal feature;

[0061] A multi-modal memory matching module for performing multi-modal memory matching between the current feature and the dynamic memory feature to obtain a fused feature after matching;

[0062] A positioning module for converting the fused feature after matching through a decoder to obtain the specific location including the temperature anomaly of the power equipment.

[0063] In the third aspect, the present invention provides a computing device, including:

[0064] A memory for storing programs;

[0065] A processor for executing the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the adaptive temperature anomaly detection method are implemented.

[0066] In the fourth aspect, the present invention provides a computer-readable storage medium, including: when the program is executed by the processor, the steps of the adaptive temperature anomaly detection method are implemented.

[0067] Advantages of the present invention: The innovative adaptive fusion mechanism of the present invention is responsible for dynamically adjusting the weights of different modality data according to environmental conditions, making full use of the spatial detail information of visible light images and the temperature distribution characteristics of infrared images, and achieving complementary advantages of multi-modal data; the memory dynamic screening mechanism intelligently stores and utilizes historical effective frame features to improve system performance, especially when environmental conditions deteriorate, it can use the information of historical high-quality frames to maintain the detection effect. Combining the above technical means, the technical solution of the present invention has strong practicability and scalability, and is not only applicable to the temperature anomaly monitoring of various power system equipment, but also can be extended to the condition monitoring of other industrial equipment, having broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. Among them:

[0069] Figure 1 It is a schematic diagram of the basic process of an adaptive temperature anomaly detection method provided by an embodiment of the present invention;

[0070] Figure 2 It is a model network structure diagram of an adaptive temperature anomaly detection method provided by an embodiment of the present invention;

[0071] Figure 3 It is a specific structure diagram of the memory dynamic screening mechanism of an adaptive temperature anomaly detection method provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention with reference to the accompanying drawings of the specification. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0073] Embodiment 1

[0074] Referring to Figure 1 , an embodiment of the present invention provides an adaptive temperature anomaly detection method, including:

[0075] S100: Obtain first modality data and second modality data;

[0076] S200: Use specific encoders to extract features from the preprocessed first-modal data and second-modal data respectively, obtaining their respective first feature representations and second feature representations;

[0077] S300: Fuse the first feature representation and the second feature representation through an adaptive fusion mechanism to obtain fused-modal features;

[0078] S400: Based on the fused-modal features, screen out effective historical frame features through a memory dynamic screening mechanism to obtain the dynamic memory features of the first feature representation, the second feature representation, and the fused-modal features;

[0079] S500: Perform multimodal memory matching on the current features and the dynamic memory features to obtain the fused features after matching;

[0080] S600: Transform the fused features after matching through a decoder to obtain the specific location containing the temperature anomaly of the power equipment.

[0081] It should be noted that, aiming at the significant defects existing in the existing multimodal temperature detection technology in complex environments, especially the problem of unstable detection performance under conditions such as low light, rain and fog weather, and complex backgrounds, through an innovative feature fusion mechanism and a dynamic memory update strategy, while ensuring the detection accuracy, the detection performance of the model under various environmental conditions is significantly improved, fully considering the particularity of power system temperature detection, and specifically solving the problems encountered by traditional detection methods in practical applications.

[0082] Therefore, through the above steps S100 - S600, it can work stably in complex environments, significantly improve the monitoring efficiency and safety of power equipment, reduce the errors caused by improper path design or insufficient feature fusion, and reduce the risk of equipment damage and maintenance costs. This method not only improves the detection accuracy but also ensures the safe operation of power equipment.

[0083] Example 2, referring to Figures 2 - 3 , is an embodiment of the present invention. Based on the previous embodiment, an adaptive temperature anomaly detection method is provided, including:

[0084] In the embodiment of the present application, in step S100, the first-modal data is a visible light image, which is collected by a visible light camera, and the second-modal data is an infrared image, which is collected by an infrared camera;

[0085] In the embodiments of the present application, in step S100, in order to ensure the accurate alignment of the visible light image and the infrared image, the system regularly synchronizes the acquisition timestamps of the two devices to ensure that images of the same location are obtained at the same moment. At this time, the visible light image provides external structure and device status information, while the infrared image provides information on temperature changes.

[0086] In the embodiments of the present application, in step S100, the visible light camera and the infrared camera should be jointly installed near the power equipment to capture the overall appearance and temperature distribution of the equipment. The camera should be fixedly installed to ensure a clear view angle and no obstacles affecting the shooting effect. According to the on-site lighting and temperature characteristics of the equipment, adjust the parameter settings of the equipment, such as exposure, focus, focal length, and photosensitivity. The infrared camera needs to have high resolution and high sensitivity to capture subtle temperature differences. All imaging devices should be connected to the central control room through a network to ensure the real-time transmission of video streams and image data. At the same time, the system needs to be configured with a storage server to save historical images for subsequent data analysis and model training.

[0087] In the embodiments of the present application, in step S100, infrared cameras and visible light cameras are reasonably deployed around the power equipment to ensure a clear and complete monitoring view of the equipment. The infrared cameras use high-resolution and high-sensitivity devices to capture subtle temperature changes; the visible light cameras optimize the settings of parameters such as exposure and focus according to the characteristics of the on-site environment. The data collected by the system includes visible light image sequences and infrared image sequences. Through a specially designed time synchronization mechanism, the precise alignment of different modality data is ensured, laying a foundation for subsequent feature fusion.

[0088] In the embodiments of the present application, in step S200, before entering the model training stage, the collected image data needs to go through a series of preprocessing steps to ensure that the model can process this data efficiently and accurately. Since the shooting angles and resolutions of the camera and the infrared camera may be different, it is necessary to perform spatial alignment on the images of the two modalities, and use the first feature point matching algorithm to make the images of the two modalities correspond in the same space. For each image, crop it according to the specific location and area of the power equipment to remove unnecessary background information and reduce interference. At the same time, normalize the pixel values of the images to standardize the temperature data and the data of the visible light images to the same range for easy model processing. Finally, use historical data or experience-based rules to label the temperature anomaly regions in the infrared images, mark the high-temperature regions in red and the lower-temperature anomaly regions in blue to generate temperature anomaly labels for the supervised learning of the model.

[0089] In the embodiments of the present application, in step S200, the collected multimodal data is systematically processed, including key steps such as noise elimination and feature extraction. For visible light images, visual features such as spatial structure and texture are mainly extracted, and the detailed information in the images is captured through a multi-level feature extraction network. For infrared images, the focus is on extracting temperature distribution features. Through techniques such as temperature gradient analysis and hot spot recognition, the temperature anomalies of the device are accurately captured. At the same time, the system also standardizes the images to ensure the consistency of different modal data in the feature space.

[0090] In an alternative embodiment, the first feature point matching method in step S200 may be the SIFT feature point matching algorithm, or it may be ORB (Oriented FAST and Rotated BRIEF), or it may be SURF (Speeded Up Robust Features);

[0091] In an alternative embodiment, the SIFT feature point matching algorithm includes constructing a scale space for each input image (including visible light and infrared images). This is usually achieved by applying Gaussian filters of different scales, thereby generating a series of Gaussian pyramid images. At each scale level, the Difference of Gaussian (DoG) images are calculated. Local extreme points in the DoG images are searched for as candidate key points. These points should be local maxima or minima in both the scale space and the two-dimensional image space. For each candidate key point, its position, scale, and principal direction are further refined. For each key point, the gradient direction histogram within its neighborhood is calculated, and the direction corresponding to the highest peak in the histogram is selected as the principal direction of the key point. Descriptors are generated, and the SIFT feature descriptors between two images are matched. The RANSAC (Random Sample Consensus) algorithm is used to estimate the best geometric transformation model (such as a homography matrix or an affine transformation) from the matched key point pairs to complete the spatial alignment of the two images.

[0092] In an alternative embodiment, the ORB algorithm includes using the FAST algorithm to detect key points in the image, calculating a principal direction for each detected key point, using the BRIEF algorithm to generate descriptors for each key point, performing feature point matching by comparing the descriptor similarities between two images, and using the RANSAC algorithm to estimate the best geometric transformation model (such as a homography matrix) based on the matched feature point pairs to complete the spatial alignment of the two images.

[0093] In an optional embodiment, the SURF algorithm includes using the determinant of the Hessian matrix to detect key points in an image, approximating the Gaussian filter with box filters, and calculating the main direction for each key point. SURF determines this based on the directional distribution of the Haar wavelet response, ensuring rotational invariance, describes each key point based on the wavelet response value and its direction, uses a nearest neighbor matching strategy (such as KNN or FLANN) to find the corresponding relationships between two images, and estimates the best geometric transformation model (such as a homography matrix) using the RANSAC algorithm based on the matched key point pairs to achieve the spatial alignment of the two images.

[0094] It should be noted that although SURF and ORB have obvious advantages in terms of processing speed, in scenarios that require high precision and reliability, such as temperature anomaly detection in power systems, the SIFT feature point matching algorithm adopted in the present invention is preferred due to its excellent stability and accuracy. Since SIFT has scale invariance and rotational invariance, it is particularly suitable for processing data collected by different sensors, which may vary due to factors such as perspective and lighting.

[0095] In the embodiment of the present application, in step S300, an encoder is used to extract features from the input multi-modal data. For the visible light input image, after a series of convolution operations and feature extractions, a first feature representation F RGB is generated. This encoder mainly focuses on the details and texture information in the visible light image. For the infrared input image, the infrared encoder also extracts its thermal information features to generate a second feature representation F T . The infrared image encoder focuses more on the thermal changes in the scene and the temperature features of objects. While extracting the encoder features, the adaptive fusion mechanism will selectively fuse the features of each modality through complex feature interaction operations including feature splicing, weighted addition, etc., and through an attention mechanism, and finally obtain the fused modality feature F E .

[0096] In the embodiment of the present application, in step S300, the processed visible light image and infrared image are respectively input into a deep learning network model for training. The model adopts a multi-modal fusion method, combines visible light and infrared information, and improves the ability to identify the state of power equipment and detect faults. After training, the model is deployed to a central server to process the data returned in real time by the image acquisition devices deployed around the power system. Through real-time calculation and analysis, the system can monitor the operating state of power equipment in a timely manner, discover potential problems and issue early warnings to ensure the safe and stable operation of the power system.

[0097] In the embodiments of the present application, in step S300, the mechanism can dynamically calculate the weight coefficients of different modal features according to the current environmental conditions, such as light intensity, weather conditions, background complexity, etc. Through the designed multi-level attention mechanism, the system can automatically identify and select the most valuable feature information while effectively suppressing the interference of environmental noise. In the feature fusion process, the system adopts the first feature fusion strategy to ensure optimal detection effects at different scales. The core advantage of this mechanism lies in its adaptability, which can adjust the fusion strategy in real time according to environmental changes to ensure the stability of detection performance.

[0098] In the embodiments of the present application, the adaptive fusion mechanism in step S300 includes the following steps:

[0099] Dynamically calculate the weight coefficients of the first modal data and the second modal data according to the current environmental parameters;

[0100] Calculate the attention matrices of the first modal data and the second modal data;

[0101] Adopt the first feature fusion strategy, combine the obtained weight coefficients and attention matrices to obtain the fused modal features.

[0102] In an alternative embodiment, the first feature fusion strategy in step S300 can be a multi-scale feature fusion strategy, can also be a deep convolutional neural network, or can also be sparse representation;

[0103] In an alternative embodiment, the multi-scale feature fusion strategy includes normalizing the input visible light image and infrared image, using SIFT to perform spatial alignment on the two images, using a deep learning network to respectively extract the multi-level feature representations of the visible light image and the infrared image, dynamically calculating the weight coefficients of different modal features through an environmental evaluation function according to the current environmental conditions, applying a multi-level attention mechanism to enhance key features and suppress noise, multiplying the feature maps at different scales by the corresponding attention matrices, and performing weighted summation according to the previously calculated weight coefficients. After multi-scale feature fusion, a feature representation that combines various scale information is obtained.

[0104] In an alternative embodiment, the deep convolutional neural network includes normalizing the input RGB and thermal infrared images, using a pre-trained DCNN model to respectively extract the deep features of the RGB and thermal infrared images, mapping the features of different modalities to the same dimensional space through a fully connected layer or a specific conversion layer, and applying weighted summation or other fusion strategies on the high-level features to integrate the information from the two modalities to obtain the fused features.

[0105] In an optional embodiment, the sparse representation includes learning a set of basis vectors from a large number of training samples, representing the input RGB and thermal infrared features as a linear combination of the basis vectors in the dictionary, and performing weighted fusion according to the sparse coefficients corresponding to each modal feature to obtain the fused feature.

[0106] It should be noted that although both DCNN and sparse representation are effective feature processing methods, when dealing with the problem of multi-modal data fusion in a complex environment, the multi-scale feature fusion strategy proposed by the present invention shows unique advantages due to its flexibility, self-adaptability, and sensitivity to detailed information.

[0107] In the embodiment of the present application, the dynamic calculation of the weight coefficients of the first modal data and the second modal data in step S300 is expressed as:

[0108] W RGB (t) = σ(f env (L(t), W(t), C(t)))

[0109] W T (t) = σ(f env (T(t), W(t), C(t)))

[0110] Among them, W RGB (t) and W T (t) respectively represent the weight coefficients of the RGB and thermal infrared modalities at time t, L(t) represents the current light intensity, W(t) represents the weather condition parameter, C(t) represents the scene complexity, T(t) represents

[0111] the environmental temperature, f env is the environmental evaluation function, and σ is the normalization function

[0112] In the embodiment of the present application, the calculation of the attention matrix of the first modal data and the second modal data in step S300 is expressed as:

[0113]

[0114] Among them, A RGB and A T are respectively the attention matrices of the first modal data and the second modal data, W q and W k are learnable query and key value parameter matrices, is the square root of the feature dimension.

[0115] In the embodiment of the present application, the first comprehensive feature F F in step S300 is expressed as:

[0116]

[0117] Among them, represents the splicing operation of the feature channel dimension. α(t) is an adaptive fusion coefficient, dynamically determined by environmental conditions. Conv is a convolution operation, used for feature dimension adjustment and preliminary fusion. Through this rigorous mathematical model, the present invention realizes stable feature fusion in various complex environments, significantly improving the adaptability and detection accuracy of the system.

[0118] In the embodiment of the present application, in step S400, effective historical frames are screened through a memory dynamic screening mechanism. The memory dynamic screening mechanism calculates the contribution degree of each modality historical frame to the accuracy of calculating temperature anomaly points. For frame features with low contribution degrees, they will be dynamically removed from the memory repository. For frame features with high contribution degrees, higher weights will be given to participate in the calculation of subsequent inference results, obtaining the dynamic memory features of the first feature representation, the second feature representation, and the fusion modality feature

[0119] In the embodiment of the present application, in step S400, the memory dynamic screening mechanism calculates and evaluates the contribution of each frame of image to the detection accuracy in real time by establishing a feature memory library. For high-quality frames, the system not only saves their feature information but also analyzes their typical feature patterns to guide the subsequent detection process. For frames of lower quality, the system will evaluate their usability and remove them from the feature library when necessary, ensuring that high-quality reference features are always maintained in the memory library. During the actual detection process, the system will intelligently combine the features of historical valid frames and the current frame, and through the first feature matching algorithm, achieve more accurate abnormal area localization and temperature estimation. The dynamic update mechanism of the memory library ensures that the system can continuously optimize and improve the detection performance.

[0120] In the embodiment of the present application, the memory dynamic screening mechanism in step S400 includes the following steps:

[0121] By establishing a feature memory library, calculate the contribution degree of each modality historical frame to the accuracy of calculating temperature anomaly points, obtaining the contribution degree evaluation result of each frame feature;

[0122] According to the contribution degree evaluation result of the current frame feature, set a dynamic screening threshold, dynamically update the memory library, and select relevant historical frame features;

[0123] Adopt the first feature matching algorithm to integrate the historical frame features into the current frame features, obtaining the dynamic memory features of the first feature representation, the second feature representation, and the fusion modality feature.

[0124] In an optional embodiment, the first feature matching algorithm in step S400 can be a designed feature matching algorithm, or a Siamese network, or a normalized cross-correlation NCC;

[0125] In an optional embodiment, the designed feature matching algorithm includes quantifying the contribution of each frame of features to the detection accuracy, determining a dynamic screening threshold based on the current frame and the historical features of its previous k frames, and for each frame of features, if its contribution degree is greater than the threshold, adding it to the memory bank; otherwise, removing the frame of features. Combining the current frame features and the historical features in the memory bank to generate the final fused features.

[0126] In an optional embodiment, the Siamese network includes preparing a data set containing a large number of paired samples, each pair of samples including a query image and a reference image, using the Siamese network structure, which consists of two sub-networks with shared weights, each sub-network responsible for extracting the feature representation of an image, training the network to minimize the distance between positive sample pairs and maximize the distance between negative sample pairs, using the trained Siamese network to extract the deep feature representations of the current frame and the historical frames in the memory bank respectively, evaluating the similarity between them by calculating the distance between the feature vectors of the two images, and selecting the historical frame most similar to the current frame for feature fusion to generate the final detection features.

[0127] In an optional embodiment, the Normalized Cross-Correlation (NCC) includes treating each historical frame in the memory bank as a template, using the sliding window technique on the current frame, comparing the image patches within each window with the templates in the memory bank, applying the Normalized Cross-Correlation (NCC), calculating the similarity scores at each position, selecting the position with the highest similarity score as the best matching point, and performing feature fusion accordingly to generate the final fused features.

[0128] It should be noted that compared with the Siamese network which requires a large amount of training data and computing resources, the feature matching algorithm designed by the present invention is more lightweight and easy to implement. At the same time, it avoids the high computational cost problem that the Normalized Cross-Correlation (NCC) may face, especially with higher efficiency when dealing with large-scale data.

[0129] In the embodiment of the present application, the contribution degree of each modal historical frame to the calculation accuracy of temperature anomaly points in step S400 is expressed as:

[0130] C(F i ) = exp(-·MSE(Pred(F i ), GT))·(1 - exp(-γ·Age(F i )))

[0131] Where MSE is the mean square error between the prediction result and the true label, and Pred(F i ) is the use of feature F iThe prediction result, GT is the ground truth label, Age(F i ) is the time decay factor of the feature, and λ and γ are adjustable hyperparameters;

[0132] In the embodiment of the present application, the dynamic update of the memory bank in step S400 is expressed as:

[0133] M t ={F j |C(F j )>θ t ,j∈[t - k,t]}θ t =μ·mean({C(F j )|j∈[t - k,t]})

[0134] where M t represents the memory feature bank at time t, θ t is the dynamic screening threshold, k is the number of historical frames considered, and μ is the threshold adjustment coefficient.

[0135] In the embodiment of the present application, in step S400, as Figure 3 shown, the figure shows the dynamic screening process of historical frames of visible light, infrared, and fusion modalities. The horizontal axis represents the time series, and each row represents the frame sequence of a different modality. The frames marked with red "×" are the historical frames identified by the system as having low contribution and thus removed, while the high-quality frames are retained in the memory bank for subsequent detection.

[0136] In the embodiment of the present application, in step S500 during the feature fusion stage, the system uses the following formula for fusion:

[0137]

[0138] where F M is the fused feature, F current is the current frame feature, F j is the historical feature in the memory bank, ∑(sim(:,:) is the similarity calculation function, and β is the historical feature influence weight coefficient. This precise mathematical mechanism ensures that the system can effectively screen and utilize high-quality historical features, significantly improving the detection stability and accuracy in harsh environments.

[0139] In the embodiment of the present application, in step S500 after the memory dynamic screening, the model further performs multi-modal memory matching. This step further optimizes the screened features to make them more in line with the reasoning requirements of the scenario. This process can match with the historical features stored in the memory module through an adaptive matching strategy and an attention mechanism, and this process can be expressed as

[0140] In the embodiment of the present application, after the multi-modal memory matching in step S600, the model starts to convert the matched fusion features into the final inference result through the decoder. The decoder uses operations such as attention mechanism, feature splicing, convolution, activation, etc., and simultaneously receives the skip connection of the fusion features from the previous encoder to obtain low-level features, and finally generates the corresponding output result and feature map.

[0141] It should be noted that through systematic technological innovation, the present invention not only solves the problem of unstable performance of traditional detection methods in complex environments, but also significantly improves the overall efficiency of the system through an intelligent feature management mechanism. Its advanced technical solution and outstanding application value provide a reliable technical guarantee for the safe and stable operation of the power system, and have important engineering application value and social benefits.

[0142] It should be noted that in the actual application scenario, the present invention establishes a complete temperature anomaly detection process. When the system captures a new image frame, it first performs preprocessing and feature extraction, and then integrates multi-modal information through an adaptive fusion mechanism. The system will simultaneously query the memory feature library and make a comprehensive judgment in combination with historical experience. For the detected abnormal area, the system not only outputs its precise location, but also provides detailed temperature information and possible fault type analysis. Such comprehensive analysis results provide an important decision-making basis for equipment maintenance personnel.

[0143] It should be noted that the core innovation points of the present invention are the design of two key components: an adaptive fusion mechanism and a memory dynamic update module. The adaptive fusion mechanism is responsible for dynamically adjusting the weights of different modal data according to environmental conditions, making full use of the spatial detail information of visible light images and the temperature distribution characteristics of infrared images, and realizing the complementary advantages of multi-modal data. The memory dynamic update module improves the system performance by intelligently saving and using the features of historical valid frames, especially being able to use the information of historical high-quality frames to maintain the detection effect when the environmental conditions deteriorate.

[0144] Embodiment 3, this is an embodiment of the present invention. The difference between this embodiment and the first embodiment is that an adaptive temperature anomaly detection system is provided.

[0145] It should be noted that the technical solution of this adaptive temperature anomaly detection system belongs to the same concept as the technical solution of the above-mentioned adaptive temperature anomaly detection method. For the details not described in detail in the technical solution of the adaptive temperature anomaly detection system in this embodiment, reference can be made to the description of the technical solution of the above-mentioned adaptive temperature anomaly detection method.

[0146] An adaptive temperature anomaly detection system in this embodiment includes:

[0147] A data acquisition module for obtaining first-modal data and second-modal data;

[0148] A feature extraction module for respectively using specific encoders to extract features from the preprocessed first-modal data and second-modal data to obtain respective first feature representations and second feature representations;

[0149] An adaptive fusion module for fusing the first feature representation and the second feature representation through an adaptive fusion mechanism to obtain fused-modal features;

[0150] A memory dynamic screening module for screening effective historical frame features based on the fused-modal features through a memory dynamic screening mechanism to obtain dynamic memory features of the first feature representation, the second feature representation, and the fused-modal features;

[0151] A multi-modal memory matching module for performing multi-modal memory matching between the current feature and the dynamic memory features to obtain fused features after matching;

[0152] A positioning module for converting the fused features after matching through a decoder to obtain the specific location including the temperature anomaly of the power equipment.

[0153] This embodiment also provides an electronic device applicable to a situation of an adaptive temperature anomaly detection method, including:

[0154] A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the adaptive temperature anomaly detection method proposed in the above embodiment.

[0155] This embodiment also provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the adaptive temperature anomaly detection method proposed in the above embodiment.

[0156] The storage medium proposed in this embodiment and the adaptive temperature anomaly detection method proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0157] From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general-purpose hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk or optical disc of a computer, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

Claims

1. An adaptive temperature anomaly detection method, characterized in that, Including: Obtain the first modal data and the second modal data; For the preprocessed first modal data and second modal data, respectively use specific encoders to extract features, obtaining the respective first feature representations and second feature representations; Fuse the first feature representation and the second feature representation through an adaptive fusion mechanism to obtain a fused modal feature; Based on the fused modal feature, screen effective historical frame features through a memory dynamic screening mechanism to obtain the dynamic memory features of the first feature representation, the second feature representation, and the fused modal feature; Perform multi-modal memory matching between the current feature and the dynamic memory feature to obtain the fused feature after matching; Transform the fused feature after matching through a decoder to obtain the specific location including the temperature anomaly of the power equipment.

2. The adaptive temperature anomaly detection method according to claim 1, wherein: The preprocessing includes the following steps: Adopt the first feature point matching algorithm to perform spatial alignment on the first modal data and the second modal data; Crop according to the specific location and area of the power equipment; Perform normalization processing on the first modal data and the second modal data; Use historical data to label the temperature anomaly area of the second modal data.

3. The adaptive temperature anomaly detection method according to claim 1 or 2, characterized in that: The adaptive fusion mechanism includes the following steps: Dynamically calculate the weight coefficients of the first modal data and the second modal data according to the current environmental parameters; Calculate the attention matrices of the first modal data and the second modal data; Adopt the first feature fusion strategy, combine the obtained weight coefficients and attention matrices to obtain a fused modal feature.

4. The adaptive temperature anomaly detection method according to claim 3, characterized in that: The memory dynamic screening mechanism includes the following steps: By establishing a feature memory bank, calculate the contribution degrees of each modal historical frame to the accuracy of calculating the temperature anomaly point, obtaining the dynamic memory features of the first feature representation, the second feature representation, and the fused modal feature; According to the dynamic memory feature, set a dynamic screening threshold, and dynamically update the memory bank to obtain an optimized memory feature bank.

5. The adaptive temperature anomaly detection method according to claim 4, characterized in that: The multi-modal memory matching between the current feature and the dynamic memory feature includes the following steps: Through an adaptive matching strategy, match the current frame feature with the dynamic memory features of the first feature representation, the second feature representation, and the fused modal feature; Adopt an attention mechanism to dynamically adjust the importance weights of each feature to obtain the fused feature after matching.

6. The adaptive temperature anomaly detection method according to claim 5, wherein: The dynamic calculation of the weight coefficients of the first modal data and the second modal data is expressed as: W RGB (t) = σ(f env (L(t), W(t), C(t))) W T (t) = σ(f env (T(t), W(t), C(t))) Among them, W RGB (t) and W T (t) respectively represent the weight coefficients of the RGB and thermal infrared modalities at time t, L(t) represents the current light intensity, W(t) represents the weather condition parameter, C(t) represents the scene complexity, T(t) represents the environmental temperature, f env is the environmental evaluation function, and σ is the normalization function The calculation of the attention matrices of the first modal data and the second modal data is expressed as: Among them, A RGB and A T are the attention matrices of the first-modal data and the second-modal data respectively, W q and W k are learnable query and key-value parameter matrices, is the square root of the feature dimension. The first comprehensive feature F F is expressed as: Among them, represents the splicing operation of the feature channel dimension, α(t) is the adaptive fusion coefficient, and Conv is the convolution operation.

7. The adaptive temperature anomaly detection method according to claim 6, wherein: The calculation of the contribution degrees of each modal historical frame to the accuracy of calculating the temperature anomaly point is expressed as: C(F i ) = exp(-·MSE(Pred(F i ), GT))·(1 - exp(-γ·Age(F i ))) where MSE is the mean squared error between the prediction result and the true label, Pred(F i ) is the prediction result using the feature F i , GT is the true label, Age(F i ) is the time decay factor of the feature, and λ and γ are adjustable hyperparameters; The dynamic update of the memory bank is expressed as: M t = {F j | C(F j ) > θ t , j ∈ [t - k, t]}θ t = μ · mean({C(F j ) | j ∈ [t - k, t]}) Among them, M t represents the memory feature library at time t, θ t is the dynamic screening threshold, k is the number of historical frames considered, and μ is the threshold adjustment coefficient.

8. An adaptive temperature anomaly detection system, which applies the method according to any one of claims 1-7, characterized in that, Including: A data acquisition module for obtaining the first modal data and the second modal data; A feature extraction module for, for the preprocessed first modal data and second modal data, respectively using specific encoders to extract features, obtaining the respective first feature representations and second feature representations; An adaptive fusion module for fusing the first feature representation and the second feature representation through an adaptive fusion mechanism to obtain a fused modal feature; A memory dynamic screening module, configured to screen effective historical frame features through a memory dynamic screening mechanism based on the fused modality features, so as to obtain a first feature representation, a second feature representation, and a dynamic memory feature of the fused modality features; A multi-modal memory matching module, configured to perform multi-modal memory matching on the current feature and the dynamic memory feature to obtain a fused feature after matching; A positioning module, configured to convert the fused feature after matching through a decoder to obtain a specific location including the temperature anomaly of the power equipment.

9. A computing device, characterized in that, Comprising: A memory, configured to store a program; A processor, configured to load the program to execute the steps of the adaptive temperature anomaly detection method according to any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, the steps of the adaptive temperature anomaly detection method according to any one of claims 1-7 are implemented.

Images