A thermal flow field anomaly detection method based on thermal imaging sequences

Abstract

This invention discloses a method for detecting thermal flow field anomalies based on thermal imaging sequences, belonging to the field of image processing and anomaly detection technology. The method includes five core steps: temporal acquisition and preprocessing, spatiotemporal feature extraction, thermal flow field reconstruction, anomaly detection, and anomaly localization output. It acquires thermal imaging video sequences through continuous temporal acquisition and performs correction processing. A three-dimensional convolutional neural network is used to jointly encode spatial temperature features and temporal evolution features. The thermal flow vector field is reconstructed based on temperature differences and optical flow estimation. A baseline model of normal operating conditions is constructed, and a contrastive learning distance metric is used to identify various anomaly types. A visualized thermal map and detection report are generated. Each step forms a deeply coupled closed-loop collaborative system. Anomaly detection results are fed back to optimize feature extraction parameters. This invention can accurately capture dynamic features of heat conduction and precisely identify anomalies such as local overheating, poor heat dissipation, heat leakage, and insulation failure.

Description

Technical Field

[0001] This invention relates to the fields of image processing and anomaly detection technology, specifically to a thermal flow field anomaly detection method based on thermal imaging sequences, which is mainly applied to thermal flow field monitoring and anomaly diagnosis of industrial equipment and building structures. Background Technology

[0002] Thermal imaging technology detects the infrared radiation emitted by a target object and converts the thermal signal into a visual image. It has been widely used in fields such as industrial equipment fault diagnosis, building structure inspection, and power equipment inspection. Traditional thermal imaging anomaly detection methods mainly rely on the analysis of single-frame thermal images, determining whether there is a fault in the equipment or structure by setting temperature thresholds or identifying temperature anomalies. However, a single-frame thermal image can only reflect the static temperature distribution at a certain moment, making it difficult to capture the dynamic process of heat conduction and the temporal evolution of the heat flow field.

[0003] Existing infrared thermal imaging anomaly detection methods suffer from the following technical problems: First, single-frame detection mode cannot acquire dynamic characteristics of heat conduction, resulting in insufficient ability to identify slowly developing thermal anomalies; second, traditional threshold methods rely on manually set threshold parameters, which have poor adaptability to complex thermal environments and are prone to false alarms and missed alarms; third, existing methods lack analysis of the direction and speed of heat propagation, making it impossible to determine the location and diffusion trend of abnormal heat sources; fourth, their ability to distinguish between multiple anomaly types is limited, making it difficult to accurately identify different anomaly modes such as local overheating, poor heat dissipation, and heat leakage.

[0004] Taking the Chinese invention patent "A Real-time Detection Method for Infrared Thermal Imaging Anomalies in Substation Equipment" (application number 202011058516.3) as an example, this method employs a lightweight deep learning scheme combining the EfficientNet network and the YOLOv3 model. It uses a target detection algorithm to locate abnormal regions and estimate temperatures in single-frame infrared images. However, this method has the following main shortcomings: First, it relies solely on single-frame images for detection, failing to utilize temporal information to analyze the dynamic process of heat conduction, making it difficult to effectively identify thermal anomalies that require time to accumulate. Second, the target detection framework for anomaly localization primarily focuses on spatial location information, lacking analysis of the direction and speed of heat flow propagation, thus failing to predict the development trend of anomalies. Third, temperature calculation is based solely on a linear mapping of grayscale values, neglecting the physical characteristics of thermal radiation and the influence of environmental factors, resulting in limited temperature measurement accuracy. Fourth, anomaly type identification mainly relies on the classification confidence of the predicted bounding box, lacking in-depth analysis of the physical mechanisms of heat conduction, and lacking sufficient ability to identify complex thermal field anomalies.

[0005] To address the aforementioned technical issues, there is an urgent need to develop a method for detecting thermal flow field anomalies that can utilize thermal imaging sequences to analyze the dynamic characteristics of heat conduction, reconstruct the spatiotemporal evolution of the thermal flow field, and accurately identify various types of anomalies. Summary of the Invention

[0006] The purpose of this invention is to provide a thermal flow field anomaly detection method based on thermal imaging sequences, which solves the technical problems of existing technologies, such as the difficulty of capturing dynamic features of heat conduction by single-frame thermal imaging and the low recognition rate of complex thermal field anomalies by traditional threshold methods, and realizes accurate monitoring of thermal flow fields of industrial equipment and building structures and accurate identification of various anomaly types.

[0007] To achieve the aforementioned objectives, the present invention employs the following technical solution: a method for detecting thermal flow field anomalies based on thermal imaging sequences. This method involves acquiring a corrected temporal image sequence through temporal acquisition and preprocessing steps; using a three-dimensional convolutional neural network to jointly encode spatial temperature features and temporal evolution features to generate spatiotemporal fusion features through a spatiotemporal feature extraction step; reconstructing the thermal flow vector field based on temperature differences and optical flow estimation through a thermal flow field reconstruction step; constructing a normal operating condition baseline model through an anomaly detection step and using a contrastive learning distance metric to identify various anomaly types; and generating a visualized thermal map and detection report through an anomaly location output step. This invention forms a deeply coupled closed-loop collaborative system comprising five core steps: temporal acquisition, spatiotemporal extraction, thermal flow reconstruction, anomaly detection, and location output. The output of each step serves as a key input for subsequent steps, and the anomaly detection results are used to optimize feature extraction parameters, achieving accurate detection and intelligent diagnosis of thermal flow field anomalies.

[0008] Compared with the prior art, the present invention has the following beneficial effects:

[0009] First, this invention achieves precise capture of the dynamic process of heat conduction through continuous temporal acquisition and joint encoding by a three-dimensional convolutional neural network. Compared with single-frame detection methods, it can identify slowly developing thermal anomalies and significantly improves the detection sensitivity for early faults. The spatiotemporal feature extraction step extracts temperature gradients and hot spot morphology in the spatial dimension and captures the heat conduction rate and temperature rise curve evolution in the temporal dimension. By adaptively adjusting feature weights through a spatiotemporal attention mechanism, it effectively enhances the feature representation capability of complex thermal fields.

[0010] Secondly, this invention calculates the heat flow vector field through a heat flow field reconstruction step, enabling quantitative analysis of the direction and velocity of heat propagation. Compared to traditional methods that only focus on temperature distribution, this method can track heat propagation paths and predict abnormal diffusion trends, providing richer physical information for fault diagnosis. The heat flow field reconstruction combines temperature differences and optical flow estimation, fully utilizing the temporal correlation between adjacent frames, significantly improving the accuracy and stability of the heat flow field reconstruction.

[0011] Third, this invention constructs a multi-center baseline model under normal operating conditions and uses a contrastive learning distance metric for anomaly detection. Compared to the traditional threshold method, this method can adapt to different operating conditions and environmental conditions, reducing false alarms and missed alarms. The baseline model is obtained through statistical analysis of normal operating condition samples, which can accurately characterize the feature distribution range of the normal state and has high sensitivity to identify anomalies that deviate from the normal pattern.

[0012] Fourth, this invention establishes discrimination rules for various anomaly types based on the spatiotemporal characteristics and physical mechanisms of the thermal flow field. These rules can accurately distinguish typical anomalies such as localized overheating, poor heat dissipation, heat loss due to leakage, and insulation failure, significantly improving the anomaly type identification capability compared to existing methods. The anomaly type identification comprehensively considers multi-dimensional information such as temperature evolution trends, heat flow vector distribution, and spatial propagation characteristics, ensuring the accuracy and reliability of anomaly diagnosis.

[0013] Fifth, this invention constructs a closed-loop feedback optimization mechanism. Anomaly detection results are fed back to adjust the attention weights for spatiotemporal feature extraction, adaptively optimizing feature extraction capabilities based on detected anomaly patterns, thereby continuously improving detection performance. This closed-loop mechanism enables the system to dynamically adjust according to actual detection conditions, optimizing computational efficiency while maintaining detection accuracy. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall process of the thermal flow field anomaly detection method of the present invention.

[0015] Figure 2 This is a flowchart illustrating the timing acquisition and preprocessing steps of the present invention.

[0016] Figure 3 This is a schematic flowchart of the thermal flow field reconstruction steps of the present invention.

[0017] Figure 4 This is a flowchart illustrating the anomaly detection steps of the present invention. Detailed Implementation

[0018] Please refer to the attached document. Figures 1-4 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown, the present invention provides a thermal flow field anomaly detection method based on thermal imaging sequences, including a time-series acquisition and preprocessing step 1, a spatiotemporal feature extraction step 2, a thermal flow field reconstruction step 3, an anomaly detection step 4, and an anomaly localization and output step 5. These steps are deeply coupled, with the output of the previous step serving as a key input parameter for the next step. The detection result of the anomaly detection step 4 is fed back to the spatiotemporal feature extraction step 2 for parameter optimization, forming a closed-loop collaborative system.

[0020] like Figure 2 As shown, step 1 of the time-series acquisition and preprocessing involves continuously acquiring data from the monitored target using an infrared thermal imager to obtain a thermal imaging video sequence. The preferred operating band for the infrared thermal imager is the 8-14μm long-wave infrared band, which corresponds to the atmospheric window, has high atmospheric transmittance, and is suitable for long-distance monitoring. The preferred resolution of the thermal imager is 640×480 pixels or higher, and the preferred frame rate is 25-60fps to ensure the capture of the temporal evolution details of the heat conduction process.

[0021] During the data acquisition process, the infrared thermal imager maintains a relatively fixed position and angle with the monitored target. The acquisition distance is determined based on the size and temperature range of the monitored target, typically ranging from 3 to 10 meters. The optimal continuous acquisition duration is 30-120 seconds, corresponding to 750-7200 frames of images. This ensures sufficient temporal information while avoiding excessive data volume that could impact processing efficiency. Environmental parameters such as ambient temperature, humidity, and wind speed are recorded during the acquisition process for subsequent temperature drift compensation.

[0022] Non-uniformity correction and temperature drift compensation are performed on each frame of the acquired thermal imaging video sequence. Non-uniformity correction eliminates image non-uniformity caused by inconsistent response characteristics among pixels in the detector array. A two-point correction method is adopted, establishing a linear correction relationship based on the response values ​​of the blackbody radiation source at low and high temperatures. Specifically, a low temperature of 15℃ and a high temperature of 85℃ are selected near the ambient temperature, and the blackbody is imaged separately to obtain the response values ​​of each pixel at the two temperature points, establishing a linear correction curve. For each pixel in the acquired image, the corrected temperature value is calculated based on its original response value and the correction curve, thus achieving non-uniformity correction.

[0023] Temperature drift compensation eliminates the impact of ambient temperature fluctuations on the detector's response characteristics. The detector's built-in temperature sensor monitors its temperature in real time. When the detector temperature changes, the image grayscale value is dynamically compensated according to a pre-calibrated temperature coefficient. The temperature coefficient is calibrated by imaging the same isothermal blackbody under different ambient temperatures and analyzing the relationship between the detector temperature and the response value. The formula for calculating temperature drift compensation is: the corrected grayscale value equals the original grayscale value plus the product of the temperature coefficient and the change in detector temperature. Through temperature drift compensation, the impact of ambient temperature fluctuations within ±10℃ on temperature measurement accuracy is eliminated, ensuring that the temperature measurement error is less than ±2℃.

[0024] After non-uniformity correction and temperature drift compensation, a corrected time-series image sequence is generated. The accuracy and consistency of temperature measurement in each frame of the sequence are significantly improved, providing high-quality input data for subsequent spatiotemporal feature extraction and thermal flow field reconstruction.

[0025] Step 2, the spatiotemporal feature extraction step, inputs the corrected temporal image sequence into a 3D convolutional neural network to achieve spatiotemporal joint encoding. The 3D convolutional neural network is a deep learning model specifically designed for processing video sequences, capable of performing convolution operations simultaneously in both spatial and temporal dimensions to extract spatiotemporal features.

[0026] The input to the 3D convolutional neural network is N consecutive frames of thermal imaging images. The preferred value for N is 16-32 frames, corresponding to a time window of 0.5-1s. This time window can capture the short-term dynamics of heat conduction while maintaining computational efficiency. The input tensor has the dimension of N×H×W×1, where N is the number of frames, H is the image height, W is the image width, and 1 represents a single-channel grayscale image.

[0027] The 3D convolutional neural network consists of multiple 3D convolutional layers and pooling layers. The first 3D convolutional layer uses 64 convolutional kernels, each with a size of 3×3×3. The first two dimensions correspond to the spatial dimension, and the third dimension corresponds to the temporal dimension. The convolutional kernels extract local patterns of temperature distribution in the spatial dimension, such as temperature gradients, edges, and textures, and extract the rate of temperature change between adjacent frames in the temporal dimension. Batch normalization and ReLU activation are performed after the convolutional operation to enhance the network's non-linear expressive power.

[0028] The first pooling layer uses max pooling with a kernel size of 2×2×2. It downsamples simultaneously in both time and space, reducing the feature map size to 1 / 8 of its original size, thus reducing computation and expanding the receptive field. Subsequent layers are a second 3D convolutional layer, a second pooling layer, a third 3D convolutional layer, and a third pooling layer. The number of convolutional kernels increases to 128 and 256 respectively, while the kernel size remains 3×3×3, and the pooling kernel size remains 2×2×2.

[0029] After multiple convolutional and pooling operations, the network extracts multi-scale spatiotemporal features. In the spatial dimension, shallow features correspond to local details of temperature distribution, such as small-scale temperature gradients and hotspot morphology, while deep features correspond to global temperature distribution patterns and large-scale thermal field structures. In the temporal dimension, shallow features correspond to short-term temperature change rates, while deep features correspond to long-term temperature rise curve evolution patterns and overall trends in heat conduction.

[0030] A spatiotemporal attention mechanism is embedded in the feature extraction process of a 3D convolutional neural network to adaptively adjust the weights of spatiotemporal features at different scales. The spatiotemporal attention mechanism includes a channel attention module and a spatial attention module, which perform attention weighting in the feature channel dimension and spatial dimension, respectively.

[0031] The channel attention module first performs global average pooling and global max pooling operations on the 3D feature maps. Global average pooling averages the feature maps in both spatial and temporal dimensions, compressing the 3D feature map of each channel into a scalar representing the global average response intensity of that channel's feature. Global max pooling extracts the maximum response value of each channel's feature, representing the peak intensity of that channel's feature. Average pooling focuses on the overall distribution of channels, while max pooling focuses on the salient features of channels; combining the two provides a more comprehensive characterization of the importance of channel features.

[0032] The feature vectors generated by global average pooling and global max pooling are respectively input into a shared multilayer perceptron network. The multilayer perceptron network consists of two fully connected layers. The first fully connected layer reduces the dimension of the feature vector to 1 / 16 of its original size. After ReLU activation, the second fully connected layer restores the dimension to the original number of channels. The multilayer perceptron network learns the non-linear dependencies between channels and outputs channel attention weight vectors.

[0033] The channel attention weight vector is calculated using the following innovative algorithm:

[0034] ,

[0035] in, Here is the channel attention weight vector. The sigmoid activation function is used, MLP is a multilayer perceptron network, GAP is global average pooling, GMP is global max pooling, and F is the input 3D feature map. The sigmoid function maps weights to the 0-1 range, representing the importance of each channel's features. The addition operation combines information from average pooling and max pooling, comprehensively considering the global distribution and salient features of the channels.

[0036] Channel attention weight vectors are multiplied channel-by-channel with the original 3D feature map to achieve channel weighting. For important feature channels, the weight is close to 1, and the original features are preserved; for unimportant feature channels, the weight is close to 0, and the original features are suppressed. The channel-weighted feature map enhances the key channels related to thermal flow anomalies and suppresses redundant and noisy channels.

[0037] The spatial attention module further weights the channel-weighted feature maps in the spatial dimension. First, average pooling and max pooling are performed in the channel dimension to compress the multi-channel feature map into two single-channel feature maps, representing the average and maximum feature intensities at each spatial location, respectively. The two single-channel feature maps are then concatenated in the channel dimension and input into a convolutional layer for spatial feature extraction. The convolutional layer uses a 7×7×1 kernel and performs convolution only in the spatial dimension, outputting a single-channel spatial attention weight map.

[0038] The spatial attention weight map is calculated using the following innovative algorithm:

[0039] ,

[0040] in, This is a spatial attention weight map. It is the Sigmoid activation function. This is a 7×7 convolution operation. Average pooling is performed along the channel dimension. Max pooling for the channel dimension, This is the channel-weighted feature map, with square brackets indicating feature concatenation operations. Each spatial location in the spatial attention weight map corresponds to a weight value between 0 and 1, representing the importance of the feature at that location.

[0041] Spatial weighting is achieved by multiplying the spatial attention weight map and the channel weighted feature map position by position. Important spatial locations, such as anomalous heat source regions and boundaries with drastic temperature gradient changes, receive higher weights, thus enhancing the original features. Conversely, less important spatial locations, such as stable and uniform background regions, receive lower weights, suppressing the original features. The spatially weighted feature map highlights key spatial regions related to thermal flow anomalies.

[0042] The final spatiotemporal fusion feature is generated through dual weighting of channel attention and spatial attention. The spatiotemporal fusion feature undergoes adaptive weighting in both the feature channel and spatial location dimensions, which strengthens the spatiotemporal features most valuable for anomaly detection, suppresses redundant and noisy features, and significantly improves the accuracy and robustness of subsequent anomaly detection.

[0043] The spatiotemporal fusion features are reduced to a one-dimensional feature vector through global average pooling, with a length of 256-512. This vector encodes the spatiotemporal joint features of the input time-series image sequence, including both spatial information such as temperature distribution and hot spot morphology, and temporal information such as heat conduction rate and temperature rise curve evolution. This feature vector is used as the output of the spatiotemporal feature extraction step 2 and is input into the thermal flow field reconstruction step 3 and the anomaly detection step 4.

[0044] like Figure 3 As shown, step 3 of the heat flow field reconstruction calculates the heat flow vector field based on the temperature difference between adjacent frames in the corrected time-series image sequence, tracking the direction and velocity distribution of heat propagation. Heat flow field reconstruction is one of the core innovations of this invention, providing rich physical information for anomaly detection through quantitative analysis of the dynamic process of heat propagation.

[0045] The first step in thermal flow field reconstruction is to calculate the temperature difference between adjacent frames. For two frames at times t and t+Δt, the temperature change at each pixel location is calculated. Let the temperature of the pixel at time t be T1, and the temperature at time t+Δt be T2. Then the rate of temperature change is the temperature difference divided by the time interval Δt. The rate of temperature change reflects the rate of heat accumulation or dissipation at that location and is a fundamental physical quantity for analyzing heat conduction processes.

[0046] The second step in thermal flow field reconstruction is to calculate the spatial temperature gradient. In each frame of the image, the temperature gradient in the x and y directions is calculated for each pixel location. The temperature gradient is calculated using the Sobel operator or the central difference method. The Sobel operator can simultaneously achieve smoothing and differentiation, reducing the influence of noise. The x-direction temperature gradient represents the rate of temperature change in the horizontal direction, and the y-direction temperature gradient represents the rate of temperature change in the vertical direction. The direction of the temperature gradient points in the direction of increasing temperature, and the magnitude of the gradient indicates the degree of temperature change.

[0047] The third step in reconstructing the heat flux field is to calculate the heat flux density vector based on the heat conduction equation. According to Fourier's law of heat conduction, the heat flux density vector is proportional to the temperature gradient, with the proportionality constant being the thermal conductivity of the material. In the image plane, the x and y components of the heat flux density vector are proportional to the temperature gradients in the x and y directions, respectively. It is important to note that the direction of heat flow is opposite to the direction of the temperature gradient; heat flows from high-temperature regions to low-temperature regions.

[0048] The heat flux density vector is calculated using the following innovative algorithm:

[0049] ,

[0050] in, Let be the heat flux density vector at position coordinates at time t, k be the local thermal conductivity, T be the temperature gradient operator, T be the temperature field, and T / t be the temperature time derivative. The optical flow vector is the motion compensation coefficient. The first term of this algorithm is the classic Fourier heat conduction term, and the second term is an innovative motion compensation term used to correct the influence of temperature field changes caused by the motion of the target or sensor on the calculation of the heat flow field.

[0051] The detailed definitions of each symbol in the formula are as follows: This is a heat flux density vector, with units of W / m³. 2 The direction represents the direction of heat propagation, and the magnitude represents the heat flow power per unit area; x and y are the spatial coordinates of the image plane, in pixels; t is time, in seconds; k is the local thermal conductivity, in W / (m·K), which reflects the thermal conductivity of the material. The k value is different for monitoring targets of different materials and can be queried through material databases or determined experimentally. The temperature gradient operator is represented in the two-dimensional image plane as follows: T is the temperature field function, with units of °C or K; T / t is the temperature-time derivative, with units of °C / s or K / s, representing the rate of change of temperature over time; It is an optical flow vector, with units of pixels per frame, representing the motion displacement of each pixel in the image between adjacent frames; The motion compensation coefficient is a dimensionless parameter, preferably ranging from 0.1 to 0.5, used to adjust the weight of the motion compensation term.

[0052] In practical applications, the estimation of local thermal conductivity k is based on the material properties of the monitored target. For metallic materials, thermal conductivity is high, typically 50-400 W / (m·K); for thermal insulation materials, thermal conductivity is low, typically 0.02-0.2 W / (m·K); and for concrete, thermal conductivity is 1-2 W / (m·K). The thermal conductivity value can be set based on the known material information of the monitored target, or the equivalent thermal conductivity can be determined through calibration experiments. Motion compensation coefficient. The optical flow vector amplitude is adaptively adjusted based on the significance of the target motion. Take the larger value when the optical flow vector amplitude is small. Choose the smaller value to ensure the effectiveness of motion compensation.

[0053] The fourth step in thermal flow field reconstruction is to obtain the motion vector field using an optical flow estimation algorithm. Optical flow is the apparent motion of pixels in an image sequence, reflecting the movement of image brightness patterns. In thermal imaging sequences, optical flow reflects the movement of temperature patterns and can be used to trace the path of heat propagation. The Lucas-Kanade optical flow algorithm or the Farneback dense optical flow algorithm is used to calculate the optical flow vector field between adjacent frames.

[0054] The Lucas-Kanade algorithm, based on the assumption of constant optical flow within a local window, calculates the optical flow vector by solving a least-squares optimization problem. This algorithm is suitable for scenarios with few feature points or small motion amplitudes and offers high computational efficiency. The Farneback algorithm, based on polynomial expansion and pixel neighborhood relationships, can calculate dense optical flow fields and is suitable for complex motion scenarios. It offers high computational accuracy but requires a larger computational load. The appropriate optical flow algorithm should be selected based on the specific application scenario. For scenarios with high real-time requirements, the Lucas-Kanade algorithm is preferred; for scenarios with high accuracy requirements, the Farneback algorithm is preferred.

[0055] The optical flow vector at each position in the optical flow vector field represents the direction and distance of movement of the pixel at that position between adjacent frames. By comparing and analyzing the optical flow vector with the heat flux density vector, we can distinguish between temperature field changes caused by heat conduction and those caused by motion. When the optical flow vector and the heat flux vector are in the same direction, it indicates that the movement of the temperature field is mainly caused by motion; when the optical flow vector and the heat flux vector are not in the same direction, it indicates that the change in the temperature field is mainly caused by heat conduction.

[0056] The fifth step in thermal flux field reconstruction is to perform temporal smoothing filtering on the heat flux density vector to eliminate transient noise interference. Thermal imaging images are affected by factors such as detector noise and environmental interference, resulting in a certain amount of random noise that causes fluctuations in the calculated heat flux vector. A temporal filtering method is used to smooth the heat flux vectors of multiple consecutive frames to extract a stable trend in the evolution of the thermal flux field.

[0057] Temporal smoothing filtering employs either a sliding window averaging method or an exponentially weighted moving average method. The sliding window averaging method calculates the arithmetic mean of the heat flow vectors from several frames before and after the current frame, with a window length preferably between 5 and 9 frames, corresponding to a time window of 0.2 to 0.4 seconds. The exponentially weighted moving average method assigns exponentially decaying weights to historical heat flow vectors, with a smoothing coefficient preferably between 0.8 and 0.95. A larger smoothing coefficient retains more historical information and improves the smoothing effect, while a smaller smoothing coefficient responds faster to current changes and is suitable for rapidly changing scenarios.

[0058] After temporal smoothing filtering, stable thermal flow field reconstruction data is obtained. The thermal flow field reconstruction data includes heat flux density vectors at each time and location, which fully describes the direction, velocity, and temporal evolution of heat propagation on the monitored target surface. The thermal flow field reconstruction data, as the output of thermal flow field reconstruction step 3, is input into anomaly detection step 4 along with the spatiotemporal fusion features.

[0059] like Figure 4 As shown, step 4 of the anomaly detection process constructs a baseline model of the spatiotemporal characteristics of the thermal flow field under normal operating conditions. It uses a contrastive learning distance metric to identify anomalies deviating from the normal pattern and determines the anomaly type based on the thermal flow field characteristics. Anomaly detection is the core function of this invention, achieving accurate identification of various anomaly types by integrating spatiotemporal features and thermal flow field information.

[0060] The construction of the baseline model begins with collecting thermal imaging video sequences under normal operating conditions as training samples. Normal operating conditions refer to the normal operation of the monitored target under fault-free and anomaly-free conditions, corresponding to a stable thermal flux field distribution. The collection of training samples covers different environmental conditions, load conditions, and time periods to ensure that the baseline model can adapt to the range of variations under normal operating conditions. The optimal number of training samples is 100-500 video sequences, each 30-120 seconds long, totaling tens of thousands to hundreds of thousands of frames to provide sufficient statistical samples.

[0061] For each video sequence in the training samples, spatiotemporal fusion features are extracted through step 2 of spatiotemporal feature extraction. Each video sequence corresponds to a spatiotemporal fusion feature vector with dimensions of 256-512. The feature vectors of all normal operating condition samples are used to construct a feature dataset, which reflects the distribution of normal operating conditions in the feature space.

[0062] Cluster analysis is performed on the feature dataset to divide the feature space into multiple normal operating condition sub-regions. The cluster analysis employs either the K-means clustering algorithm or the DBSCAN density-based clustering algorithm. The K-means algorithm divides the feature space into K clusters, each corresponding to a sub-pattern of a normal operating condition. The value of K is determined based on the complexity of the actual operating conditions, preferably between 3 and 10. The DBSCAN algorithm automatically determines the number and shape of clusters based on density connectivity, making it suitable for scenarios with irregular distributions of normal operating conditions.

[0063] After cluster analysis, the central eigenvector and characteristic distribution range of each normal operating condition sub-region are calculated. The central eigenvector is the average of all sample eigenvectors within that sub-region, representing the typical characteristics of that sub-region. The characteristic distribution range is determined by calculating the maximum distance or standard deviation of the sample eigenvectors within that sub-region to the center, representing the range of variation of normal operating conditions within that sub-region. The central eigenvector and distribution range of each sub-region are stored as baseline model parameters.

[0064] After the baseline model is constructed, spatiotemporal fusion features are extracted from the temporal image sequences to be detected. A contrastive learning distance metric is then used to calculate the deviation between the detected features and the baseline model. Two methods are used: Mahalanobis distance and Euclidean distance.

[0065] Mahalanobis distance is a distance metric that considers the correlation between feature variables. The formula for calculating it is:

[0066] ,

[0067] in, The Mahalanobis distance, The spatiotemporal fusion feature vector to be detected. The feature vector of the center of the sub-region of the baseline model. The characteristic covariance matrix, Let T be the inverse of the covariance matrix, and the superscript T denotes the vector transpose. Mahalanobis distance eliminates the correlation and dimensionality effects between feature variables, and has better robustness for high-dimensional feature spaces.

[0068] Euclidean distance is the most commonly used distance metric, and its calculation formula is:

[0069] ,

[0070] in, For Euclidean distance, Let be the spatiotemporal fusion feature vector to be detected, and its i-th component be . , The feature vector of the center of the sub-region of the baseline model is given by the i-th component. d is the dimension of the feature vector. Euclidean distance is simple to calculate and is well-suited for situations where the correlation between feature variables is weak.

[0071] Calculate the distance from the feature vector to be detected to the center of each normal operating condition sub-region, and select the minimum distance as a measure of deviation. Set a deviation threshold as a certain multiple of the baseline model distribution range; when the minimum distance exceeds the deviation threshold, it is judged as an anomaly. The deviation threshold is determined according to the sensitivity requirements of anomaly detection, preferably 2-3 times the distribution range. Setting the deviation threshold too small will lead to an increased false alarm rate, while setting it too large will lead to an increased false negative rate; it is necessary to balance the risks of false alarms and false negatives based on the actual application.

[0072] Once an anomaly is identified, the anomaly type is further determined. Anomaly type identification is based on a comprehensive analysis of reconstructed thermal flow field data and spatiotemporal characteristics, establishing discrimination rules for various anomaly types. This invention identifies four typical thermal flow field anomalies: localized overheating, poor heat dissipation, heat loss due to leakage, and insulation failure.

[0073] The criteria for identifying localized overheating anomalies are as follows: a sustained temperature rise in a localized area is detected, and heat flux vectors are concentrated in that area. A sustained temperature rise is determined by analyzing the temperature time-series curves of the area in multiple consecutive frames of images. When the temperature rises monotonically within a certain time window and the increase exceeds a set threshold, it is considered a sustained temperature rise. Heat flux vector concentration is determined by analyzing the heat flux vector distribution around the area. When heat flux vectors from multiple surrounding locations point towards this area and their amplitudes are large, it is considered a concentrated heat flux vector. Localized overheating is typically caused by internal equipment malfunctions, frictional heating, current overload, etc., and is an important indicator of equipment abnormalities.

[0074] The criteria for identifying poor heat dissipation are as follows: a localized area is detected where the temperature is higher than the surrounding area, and the heat flux vector dissipation velocity is lower than normal. The higher temperature is determined by calculating the temperature difference between this area and its neighboring regions; when the temperature difference exceeds a set threshold, it is considered a localized high temperature. The heat flux vector dissipation velocity is determined by analyzing the amplitude of the heat flux vector emanating from this area; when the amplitude is significantly lower than the typical value under normal operating conditions, the dissipation velocity is considered low. Poor heat dissipation is usually caused by blocked heat dissipation channels, aging heat dissipation materials, poor ventilation, etc., leading to heat accumulation and ineffective heat dissipation.

[0075] The criteria for identifying abnormal heat loss leakage are as follows: an unexpected propagation path is detected in the heat flux vector field, and the temperature continues to decrease along the path. Unexpected propagation paths are determined by comparing them with the heat flux vector field under normal operating conditions. When the direction of the heat flux vector along a path is significantly different from that under normal operating conditions, it is considered an unexpected path. Continuous temperature decrease is determined by analyzing the temperature distribution along the path. When the temperature continuously decreases from the starting point to the ending point, it is considered a heat leak. Heat loss leakage is usually caused by insulation layer damage, seal failure, pipeline leaks, etc., leading to heat loss along abnormal paths.

[0076] The criteria for identifying insulation failure are as follows: a sudden change in the heat flux vector in a localized area is detected, accompanied by a sharp increase in the temperature difference with the surrounding area. A sudden change in the heat flux vector is determined by calculating the difference between the current heat flux vector and historical heat flux vectors; when the difference exceeds a set threshold, it is considered a sudden change in the heat flux vector. A sharp increase in temperature difference is determined by analyzing the temporal changes in the temperature difference between this area and the surrounding area; when the temperature difference increases rapidly within a short period, it is considered a sharp increase in temperature difference. Insulation failure is usually caused by damage to the insulation material, loosening of the insulation structure, etc., leading to a sudden change in heat transfer characteristics.

[0077] The discrimination of each anomaly type comprehensively considers multi-dimensional information such as temperature distribution, temperature evolution over time, heat flux vector direction, and heat flux vector amplitude to ensure the accuracy of anomaly type identification. In practical applications, multiple anomalies may exist simultaneously. In this case, the primary and secondary anomaly types are output based on the degree of matching of the discrimination rules.

[0078] Anomaly detection step 4 outputs anomaly detection results, including whether an anomaly was detected, the anomaly type, the anomaly location, and the degree of deviation. These results serve as input to anomaly localization step 5 and are simultaneously fed back to spatiotemporal feature extraction step 2 for closed-loop optimization.

[0079] Step 5 of the anomaly localization and output process generates a heatmap visualization label based on the anomaly detection results and outputs a thermal flow field anomaly detection report. The heatmap uses color mapping to intuitively display the spatial distribution and severity of anomalies, making it easy for users to quickly locate anomalies and assess risks.

[0080] The heatmap is generated using a color mapping method, which maps the degree of deviation to different colors. The color mapping uses a continuous spectrum from blue to red, with blue corresponding to low deviation (normal area), green to slight deviation, yellow to moderate deviation, and red to high deviation (severe abnormal area). The color mapping is calculated based on the normalized value of the deviation, linearly mapping the deviation to the 0-1 interval, and then determining the corresponding RGB color value according to a color interpolation table.

[0081] Anomalies are categorized into three severity levels: mild, moderate, and severe, based on the degree of deviation. Mild anomalies correspond to a deviation of 1-2 times the baseline model standard deviation; these are weak anomalies, possibly in an early stage, requiring continuous monitoring. Moderate anomalies correspond to a deviation of 2-3 times the baseline model standard deviation; these are obvious anomalies, requiring timely investigation and preventative measures. Severe anomalies correspond to a deviation of more than 3 times the baseline model standard deviation; these are serious anomalies with a significant risk of failure, requiring immediate action.

[0082] The heatmap marks the spatial location of anomalous areas using bounding boxes or contour lines. Spatial location information includes the center coordinates, area size, and boundary shape of the anomalous area, facilitating precise anomaly source identification. For multiple dispersed anomalous areas, the location and severity level of each area are marked, and the number and distribution statistics of anomalous areas are output.

[0083] The temperature evolution trend curve is superimposed on the heat map to show the temperature change over time in key locations or abnormal areas. The temperature evolution trend curve is in the form of a line graph, with time on the horizontal axis and temperature on the vertical axis. The trend of the curve reflects the temperature change trend. For local overheating anomalies, the temperature evolution curve shows an upward trend; for heat loss leakage anomalies, the temperature evolution curve shows a downward trend; for poor heat dissipation anomalies, the temperature evolution curve shows a slow upward trend or a high-level plateau trend.

[0084] The heat map overlays the heat flow field propagation path, using arrow vector graphics to show the direction and magnitude of the heat flow vector. The direction of the arrow indicates the direction of heat propagation, and the length of the arrow indicates the magnitude of the heat flux density. By observing the heat flow field propagation path, users can intuitively understand the source, propagation path, and convergence point of heat, which helps in analyzing the root cause and scope of impact of anomalies.

[0085] The thermal flux anomaly detection report is output in structured text format, including the following: detection time, monitoring target name, detection result summary, anomaly type, anomaly location, anomaly severity level, deviation value, temperature statistics, thermal flux statistics, historical trend comparison, and recommended measures. The detection result summary briefly states whether an anomaly was detected and the number of anomalies. The anomaly type lists the various anomalies identified. The anomaly location provides specific spatial coordinates or a description of the area. The anomaly severity level is marked as mild, moderate, or severe. The deviation value provides the distance from the baseline model. The temperature statistics include the highest temperature, lowest temperature, average temperature, and temperature standard deviation. The thermal flux statistics include the maximum heat flux density, average heat flux density, and the dominant direction of the heat flux vector. The historical trend comparison shows a comparison between the current detection results and historical detection results. The recommended measures provide corresponding handling suggestions based on the anomaly type and severity level.

[0086] The report supports exporting to PDF or Word format for easy archiving and distribution. It embeds visual charts such as heat maps, temperature evolution curves, and thermal flow vector diagrams to enhance intuitiveness and readability. The report can also be configured to be automatically sent to a specified email address or pushed to a monitoring platform for timely notification and response to anomalies.

[0087] This invention constructs a closed-loop feedback optimization mechanism, in which the detection result of anomaly detection step 4 is fed back to spatiotemporal feature extraction step 2, and the weight parameters of the spatiotemporal attention mechanism are adjusted according to the detected anomaly type. This closed-loop feedback mechanism enables the system to adaptively optimize based on actual detection conditions, improving the detection capability for specific anomaly patterns while reducing computational burden when no anomalies are detected.

[0088] When a certain anomaly type is detected, the system analyzes the spatiotemporal feature patterns corresponding to the anomaly, identifying the feature channels and spatial regions most relevant to the anomaly. By increasing the channel attention weights of these key feature channels, the system enhances its feature extraction capability for the anomaly pattern. Specifically, the feature responses of samples with detected anomalies are compared with the feature responses of normal samples from the baseline model, and the discriminative power of each feature channel is calculated. Channels with higher discriminative power are assigned greater attention weights.

[0089] When no anomalies are detected after multiple consecutive checks, the system determines that it is in a stable and normal operating condition. It then reduces the attention given to detailed features by the spatiotemporal attention mechanism, thereby reducing the computational load of the 3D convolutional neural network. Specifically, this can be achieved by increasing the stride of the pooling layer, reducing the spatial resolution of the feature map, or reducing the number of convolutional kernels and feature channels in the convolutional layers, thus lowering computational complexity. This reduction in computational load allows the system to improve processing efficiency while maintaining basic detection capabilities, extending device uptime, or supporting parallel detection of more targets.

[0090] Upon detecting an anomaly, the system improves the accuracy of feature extraction from the anomaly region and its surrounding areas. Specifically, it uses a smaller pooling stride or no pooling at all for the anomaly region to preserve more spatial detail features, and increases the number of convolutional kernels for a certain range around the anomaly region to extract richer feature representations. This improved feature extraction accuracy helps to more accurately analyze the nature, severity, and impact range of the anomaly, providing more detailed information for subsequent fault diagnosis and handling.

[0091] The closed-loop feedback optimization mechanism uses an exponentially weighted moving average for weight adjustments to avoid drastic weight fluctuations. The new attention weight is a weighted average of the current and historical weights, with historical weights accounting for a larger proportion and current adjustments accounting for a smaller proportion, ensuring smooth weight changes. The magnitude of the weight adjustment is determined based on the confidence level of the detection results. When the confidence level of anomaly detection is high, the weight adjustment magnitude is larger; when the confidence level is low, the weight adjustment magnitude is smaller, avoiding incorrect weight adjustments due to misjudgments.

[0092] Through a closed-loop feedback optimization mechanism, this invention realizes an adaptive anomaly detection system. The system can continuously optimize the feature extraction strategy according to the actual detection situation, achieve a dynamic balance between detection accuracy and computational efficiency, adapt to different monitoring scenarios and anomaly modes, and significantly improve the practicality and robustness of thermal flow field anomaly detection.

[0093] To verify the effectiveness of this invention, an experiment was conducted in an industrial boiler thermal flow field monitoring scenario. The experimental subject was a 10MW coal-fired industrial boiler, and the monitoring locations were the outer wall of the furnace and the connection point of the flue. The monitoring objective was to detect abnormalities such as defects in the furnace wall insulation layer, flue leakage, and localized overheating. Continuous monitoring was performed using an infrared thermal imager with a resolution of 640×480 pixels and a frame rate of 30fps. The acquisition distance was 5m, and the acquisition time was 60s, corresponding to 1800 frames of images.

[0094] The experiment collected thermal imaging video sequences under normal operating conditions and various abnormal operating conditions. Normal operating conditions represented stable boiler operation without faults, with 200 samples collected to build a baseline model. Abnormal operating conditions included: localized overheating due to partial detachment of the insulation layer, high-temperature flue gas leakage due to loose flue flanges, poor heat dissipation due to circulating fan failure, and insulation failure due to aging of the insulation material; 50 samples were collected for each type of abnormality for testing.

[0095] The baseline model employs K-means clustering to divide the spatiotemporal fusion features of normal operating condition samples into five sub-regions, each corresponding to normal operating conditions under different load rates. Mahalanobis distance is used as the distance metric for contrastive learning, with a deviation threshold set to 2.5 times the distribution range. Anomaly detection results show an accuracy rate of 96% for local overheating, 94% for flue gas leakage, 92% for poor heat dissipation, and 93% for insulation failure, with an overall detection accuracy of 93.75%, significantly higher than the single-frame detection accuracy of 78% achieved by the comparative document method.

[0096] Further comparison was made between the present invention and the prior art method in terms of early anomaly detection capability. A simulated scenario was set up where a local defect in the insulation layer gradually expanded, with the defect area gradually increasing from 0.01 square meters to 0.1 square meters, and the initial detection time of the two methods was monitored. The present invention's method detected the anomaly when the defect area reached 0.03 square meters, while the prior art method only detected the anomaly when the defect area reached 0.07 square meters, indicating that the early detection sensitivity of the present invention is improved by 133%. This improvement in early detection capability is due to the present invention's time-series analysis of the dynamic process of heat conduction, which can capture the continuous upward trend of temperature and the gradual change of the heat flow field.

[0097] The false positive and false negative rates of the two methods were compared and analyzed. In a 24-hour continuous monitoring experiment, the method of this invention had 3 false positives and 1 false negative, with a false positive rate of 0.52% and a false negative rate of 0.17%. The method of the comparison document had 15 false positives and 5 false negatives, with a false positive rate of 2.60% and a false negative rate of 0.87%. The false positive and false negative rates of the method of this invention are significantly lower than those of the method of the comparison document, verifying the robustness of the anomaly detection method based on the baseline model and the contrastive learning distance metric.

[0098] The experiment also verified the effectiveness of the closed-loop feedback optimization mechanism. In the initial operation phase, the system achieved a 90% accuracy rate in detecting various anomalies. After one week of operation and closed-loop optimization, the accuracy rate increased to 94%, an improvement of 4 percentage points. Closed-loop optimization enabled the weight parameters of the spatiotemporal attention mechanism to adaptively adjust according to the actual detected anomaly types, enhancing the feature extraction capability for specific anomaly patterns.

[0099] The above experiments demonstrate that the present invention significantly outperforms existing technologies in key performance indicators such as accuracy of thermal flow field anomaly detection, early detection sensitivity, false alarm rate, and false negative rate, thus verifying the effectiveness and advancement of the technical solution of the present invention.

[0100] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting thermal flow field anomalies based on thermal imaging sequences, characterized in that, Includes the following steps: The time-series acquisition and preprocessing steps involve using an infrared thermal imager to continuously acquire thermal imaging video sequences of the monitored target, performing non-uniformity correction and temperature drift compensation on each frame of the thermal imaging video sequence, and generating a corrected time-series image sequence. The spatiotemporal feature extraction step involves inputting the corrected temporal image sequence into a three-dimensional convolutional neural network. In the spatial dimension, it extracts the temperature distribution gradient features and hot spot morphology features of each frame image. In the temporal dimension, it captures the heat conduction rate features and temperature rise curve evolution features between consecutive frames. Through a spatiotemporal attention mechanism, it adaptively adjusts the weights of spatiotemporal features at different scales to generate spatiotemporal fusion features. The heat flow field reconstruction step involves calculating a heat flow vector field based on the temperature difference between adjacent frames in the corrected temporal image sequence. The calculation of the heat flow vector field is based on the temperature difference between two adjacent frames, determining the heat propagation direction through the temperature gradient direction and the heat propagation speed through the temperature change amplitude. An optical flow estimation algorithm is used to track the direction and speed distribution of heat propagation. This optical flow estimation algorithm employs either the Lucas-Kanade algorithm or the Farneback algorithm, obtaining the motion vector field of heat propagation by tracking the displacement of the same temperature region in adjacent frames. The calculation of the heat flow field vector includes the following sub-steps: calculating the temperature time derivative based on the temperature values ​​at position and time in adjacent frames; determining the heat flow density vector based on the heat conduction equation using the temperature time derivative and the spatial temperature gradient; performing motion compensation on the heat flow density vector using the motion vector field obtained by the optical flow estimation algorithm to generate a corrected heat flow field vector; and performing temporal smoothing filtering on the corrected heat flow field vector to eliminate instantaneous noise interference and obtain stable heat flow field reconstruction data. The anomaly detection step involves constructing a baseline model of the spatiotemporal characteristics of the thermal flow field under normal operating conditions. This baseline model is obtained through statistical analysis of the spatiotemporal fusion characteristics of samples under normal operating conditions. A contrastive learning distance metric is used to calculate the degree of deviation between the spatiotemporal fusion characteristics to be detected and the baseline model. Based on the degree of deviation and the reconstructed thermal flow field data, the anomaly types of local overheating, poor heat dissipation, heat leakage, and insulation failure are identified, and anomaly detection results are generated. The anomaly location output step generates a heat map visualization annotation based on the anomaly detection results. The heat map annotates the spatial location of the anomaly area, the anomaly severity level, the temperature evolution trend, and the heat flow field propagation path, and outputs a heat flow field anomaly detection report. The detection results of the anomaly detection step are fed back to the spatiotemporal feature extraction step. The weight parameters of the spatiotemporal attention mechanism are adjusted according to the detected anomaly type to enhance the feature extraction capability for specific anomaly patterns and form a closed-loop optimization mechanism. When no anomaly is found in multiple consecutive detections, the spatiotemporal attention mechanism is reduced to focus on detailed features to reduce computation. When an anomaly is detected, the feature extraction accuracy of the anomaly region and its surrounding region is improved.

2. The method for detecting thermal flow field anomalies according to claim 1, characterized in that, In the timing acquisition and preprocessing steps, the non-uniformity correction adopts a two-point correction method, which establishes a linear correction relationship by using the response values ​​of the blackbody radiation source at low and high temperature points to eliminate the inconsistency in the response between pixels of the detector array; the temperature drift compensation process dynamically compensates the image grayscale value by monitoring the detector temperature change in real time and according to the temperature coefficient, thereby eliminating the influence of ambient temperature fluctuations on the temperature measurement accuracy.

3. The method for detecting thermal flow field anomalies according to claim 1, characterized in that, In the spatiotemporal feature extraction step, the three-dimensional convolutional neural network includes multiple three-dimensional convolutional layers and pooling layers. The convolutional kernel size of the three-dimensional convolutional layer is 3×3×3, and convolution operations are performed simultaneously in the temporal and spatial dimensions. The pooling layer uses max pooling to reduce the feature dimension. The spatiotemporal attention mechanism strengthens key spatiotemporal features and suppresses redundant information by calculating the importance weights of each channel in the feature map.

4. The method for detecting thermal flow field anomalies according to claim 1, characterized in that, In the spatiotemporal feature extraction step, the weight calculation of the spatiotemporal attention mechanism is implemented in the following way: Global average pooling and global max pooling operations are performed on the 3D feature map to generate the average feature vector and the maximum feature vector, respectively. The average feature vector and the maximum feature vector are input into a shared multilayer perceptron network to generate channel attention weights; The original 3D feature map is weighted and modulated based on the channel attention weights to generate a channel-weighted feature map. The channel-weighted feature map is subjected to average pooling and max pooling in the spatial dimension to generate spatial attention weights; The channel weighted feature map is spatially weighted based on the spatial attention weight to generate the final spatiotemporal fusion feature.

5. The method for detecting thermal flow field anomalies according to claim 1, characterized in that, In the anomaly detection step, the baseline model is constructed in the following way: Thermal imaging video sequences under normal operating conditions are collected as training samples, and spatiotemporal fusion features of the training samples are extracted. Cluster analysis is performed on the spatiotemporal fusion features to divide the feature space into multiple normal operating condition sub-regions; Calculate the central feature vector and feature distribution range of each normal operating condition sub-region, and construct a multi-center baseline model; The contrastive learning distance metric uses Mahalanobis distance or Euclidean distance to calculate the distance from the spatiotemporal fusion feature to be detected to the center of each normal operating condition sub-region. When the distance exceeds a preset threshold, it is determined to be abnormal.

6. The method for detecting thermal flow field anomalies according to claim 1, characterized in that, In the anomaly detection step, the anomaly type is identified based on the following feature discrimination rules: When a localized temperature rise is detected and the heat flow vector is concentrated in that area, it is identified as a localized overheating anomaly. When a local area is found to have a higher temperature than the surrounding area and a lower heat flow vector divergence rate than normal, it is identified as an abnormality of poor heat dissipation. When an unexpected propagation path is detected in the heat flux vector field and the temperature continues to drop along the path, it is identified as an abnormal heat loss leakage. When a sudden change in the heat flux vector in a local area is detected and the temperature difference with the surrounding area increases sharply, it is identified as an insulation failure anomaly.

7. The method for detecting thermal flow field anomalies according to claim 1, characterized in that, In the anomaly localization output step, the heatmap is generated using a color mapping method, mapping the degree of anomaly to different colors. The higher the degree of anomaly, the more the color leans towards red or bright colors, while the normal area leans towards blue or dark colors. The severity level of the anomaly is divided into three levels: mild, moderate, and severe, based on the magnitude of the deviation. Mild anomaly corresponds to a deviation of 1-2 times the standard deviation of the baseline model, moderate anomaly corresponds to 2-3 times, and severe anomaly corresponds to more than 3 times.

Images