Thermal Imaging Temperature Rise Trend Early Warning System Based on Spatiotemporal Sequence Prediction

Abstract

This invention discloses a thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction, specifically relating to the field of thermal imaging data prediction and early warning technology. It includes modules for image conversion, fluctuation feature extraction, edge prediction, anomaly characterization, and prediction decision-making. The system extracts thermal imaging temperature data of the target area to construct a spatiotemporal sequence, calculates the dynamic rate of temperature change and gradient intensity, and performs edge model prediction based on a combination of fluctuation indices. When the fluctuation indices do not exceed the stable range, a lightweight deep network model deployed at the thermal imaging acquisition end is invoked; when they exceed the stable range, the prediction decision-making module determines whether to switch to a higher-order multimodal model. This invention improves the spatiotemporal information extraction capability by constructing a temperature evolution data volume, dynamically controls the prediction path based on fluctuation indices to improve prediction stability and efficiency, and achieves adaptive switching of model invocation by combining anomaly activation index and prediction offset index, enhancing the accuracy and adaptability of the early warning system.

Description

Technical Field

[0001] This invention relates to the field of thermal imaging data prediction and early warning technology, and more specifically, to a thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction. Background Technology

[0002] With the rapid development of IoT technology and intelligent inspection equipment, infrared thermal imaging, as a non-contact and efficient detection method, has been widely applied in operational status monitoring tasks in many key scenarios such as power systems, industrial station control areas, intelligent substations, rail transit, and energy and chemical industries. This type of equipment continuously collects infrared radiation images of the target equipment surface, accurately reflecting its temperature distribution without interfering with equipment operation. This enables early identification of internal operating loads, thermal stress accumulation, and potential fault risks. Especially in scenarios with high voltage, high power, or complex electrical structures, thermal imaging monitoring is increasingly valued as a crucial component in ensuring operational safety.

[0003] However, existing infrared thermal imaging monitoring systems still face numerous technical bottlenecks in practical deployment. Most systems primarily rely on static analysis of single-frame thermal images or simple threshold judgments of current temperature values, failing to fully utilize the temporal evolution patterns inherent in consecutive multi-frame images, resulting in a severe deficiency in dynamic temperature trend prediction capabilities. These methods often only generate alarms after the temperature has exceeded limits or hot spots have clearly accumulated, making it difficult to promptly reflect the warning stages when equipment is about to enter an abnormal state, posing a risk of response lag and warning failure.

[0004] Furthermore, while some systems incorporate machine learning-based prediction models, their deployment environments are often embedded edge computing devices with limited computing power, memory, and inference capabilities. This necessitates significant model pruning, compromising both prediction stability and accuracy. This is particularly problematic under complex conditions such as drastic thermal fluctuations and sudden changes in equipment load, leading to substantial deviations in prediction results. Simultaneously, current systems typically employ fixed model strategies, lacking the ability to flexibly adjust model structure or paths based on operational status. The absence of intelligent discrimination mechanisms at the model invocation level results in slow response times when switching between stable and unstable thermal states, exhibiting significant rigidity, passivity, and insufficient adaptability. Therefore, this invention proposes a thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction to address these issues. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction includes an image conversion module, a fluctuation feature extraction module, an edge prediction module, an anomaly characterization module, and a prediction decision module.

[0007] The image conversion module acquires continuous frame images through an infrared thermal imaging device. Based on the spatial coordinates of the target area in the image and the shooting timestamp, it extracts the thermal zone temperature value in each frame and constructs a temperature evolution data volume with spatial positioning and time series attributes, forming a spatiotemporal sequence structure.

[0008] The fluctuation feature extraction module calculates the rate of temperature change per unit time and the root mean square of the gradient of hot spot pixels in the thermal image for the spatiotemporal sequence structure. It obtains the dynamic rate of temperature change and the intensity of local temperature gradient, and forms a combination of fluctuation indicators to determine whether the temperature rise trend is in a stable state.

[0009] The edge prediction module calls a lightweight deep network model deployed at the thermal imaging acquisition terminal when the volatility index combination is in a stable range. This model predicts the temperature changes of the target area in multiple future time steps and serves as an early warning basis. The prediction curves output by the first n calls of the lightweight deep network model constitute the lightweight prediction curve.

[0010] The anomaly characterization module generates an anomaly activation index based on the difference between the absolute value of the maximum hotspot gradient change in the target area and the historical average change when the combination of volatility indicators exceeds the stable range. Furthermore, it generates a prediction offset index based on the mean square residual between the lightweight prediction curve and the actual temperature rise trend data corresponding to the previous n calls to the lightweight deep network model, in order to evaluate the prediction stability of the edge model.

[0011] The prediction and decision module receives the abnormal activation index and prediction offset index as inputs, which are then fed into the trained machine learning model. Based on the judgment result of the machine learning model, it selects to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model, and outputs the temperature change trajectory of the target area in the future multiple time steps as the basis for early warning.

[0012] In a preferred embodiment, the image conversion module has an image frame preprocessing structure consisting of an image noise filtering component and a thermal zone recognition component. The image noise filtering component is used to suppress noise in the continuous frame images acquired by the infrared thermal imaging device. The thermal zone recognition component is used to automatically extract the position boundary and center reference point of the target area in the image and generate spatial coordinate data. The spatial coordinate data is mapped one-to-one with the shooting timestamp to form an image frame sequence with positioning information.

[0013] After the image frame sequence is processed by the image conversion module, the temperature value of the hot zone in each frame is extracted. Combined with spatial coordinate data and shooting timestamp parameters, a temperature evolution data volume with spatial positioning and time series attributes is generated. The temperature evolution data volume stores the temperature change information of the target area at multiple consecutive time points in a structured manner. The image conversion module finally outputs the temperature evolution data volume, which serves as the input basis for the subsequent fluctuation feature extraction module to calculate the combination of fluctuation indicators.

[0014] In a preferred embodiment, the fluctuation feature extraction module performs numerical calculations based on the temperature evolution data volume in the spatiotemporal sequence structure. The temperature dynamic change rate is obtained by dividing the difference between the mean temperatures of the target region at two adjacent time points in the temperature evolution data volume by the time interval, thus forming a numerical expression reflecting the rate of temperature change per unit time.

[0015] The local temperature gradient intensity is obtained by establishing a two-dimensional pixel window centered on the target region in each frame of the thermal image, extracting the gray value gradient of all hot spot pixels within the two-dimensional pixel window, and then taking the square root of the average of the sum of squares of all pixel gradient values ​​to obtain the root mean square value of the hot spot pixel gradient of the corresponding frame.

[0016] In a preferred embodiment, the volatility indicator combination consists of the following data one and data two:

[0017] Data 1 was obtained by averaging multiple temperature change rates within a preset fixed time window;

[0018] Data 1 was obtained by finding the root mean square maximum of the local temperature gradient in multiple frames of images within a preset fixed time window;

[0019] Determining whether the temperature rise trend is in a stable state refers to:

[0020] The volatility index combination is compared with the preset stability reference index combination's value range. If the volatility index combination falls within the preset stability reference index combination's value range, the temperature rise trend is in a stable state; otherwise, the temperature rise trend is not in a stable state.

[0021] In a preferred embodiment, the lightweight deep network model deployed in the edge prediction module is constructed based on a temporal convolutional network or a long short-term memory network. The input data comes from the temperature evolution data volume output by the image conversion module. When the edge prediction module calls the lightweight deep network model, it uses the temperature value sequence of the target area in multiple consecutive time steps as the input data. The temperature value sequence includes the original thermal imaging temperature data and the first-order differential temperature change data of several frames before the current time point.

[0022] After receiving input data, the lightweight deep network model performs pre-defined feature extraction on the time series structure and generates a series of predicted temperature values ​​for multiple future time steps in the target area through a recursive approach.

[0023] In a preferred embodiment, the abnormal activation index is generated as follows:

[0024] Perform one-dimensional or two-dimensional Sobel operator operations on the temperature distribution of the target area in the current infrared thermal imaging image, calculate the temperature change value of each pixel in the gradient direction, and extract all local maximum gradient change values ​​in the target area based on a sliding window of fixed size, and select the global maximum value as the absolute value of the maximum hot spot gradient change in the current image frame.

[0025] Extract the absolute value of the maximum hot spot gradient change in the target region from the first m frames of images, and perform an arithmetic average to obtain the historical average change as a reference metric.

[0026] The deviation is obtained by subtracting the historical average change from the absolute value of the maximum hotspot gradient change in the current image frame. Then, the deviation is normalized by using the sum of the historical average change and the preset stability constant as the denominator. The stability constant is a non-zero positive number to avoid numerical instability caused by the denominator approaching zero. Finally, the normalized result is compared with zero. If the result is less than zero, it is taken as zero. If the result is greater than zero, its original value is retained. This determines the abnormal activation index.

[0027] In a preferred embodiment, the predicted offset index is generated as follows:

[0028] Obtain n lightweight prediction curves generated by the lightweight deep network model during the first n calls, and simultaneously obtain the actual temperature rise trend data corresponding to each call. Calculate the squared difference between the predicted value and the actual value for each prediction curve and its corresponding actual temperature rise trend data at multiple time steps to obtain n residual sums of squares, forming a residual sequence.

[0029] The residual sequence is divided into equal-width sliding windows. The maximum squared residual value is extracted in each window, and its frequency of occurrence in all windows is counted. The ratio of this frequency value to the total number of windows is defined as the local surge ratio. The maximum squared residual value of each window is multiplied by its corresponding local surge ratio, the sum is obtained, and then divided by the total number of windows to obtain the residual confidence weight.

[0030] Calculate the standard deviation of the squared residuals for n residuals, and calculate the difference between the maximum and minimum values. The sum of the standard deviation and the range is used as the prediction stability factor. Multiply the residual confidence weights by the prediction stability factor to obtain the prediction bias index.

[0031] In a preferred embodiment, the trained machine learning model used in the prediction decision module is a fuzzy logic device. The fuzzy logic device includes an input membership function set, a rule base, and an output inference mechanism. The input membership function set is used to receive the abnormal activation index and the prediction offset index as input parameters, and convert the two input parameters into corresponding membership values ​​based on the set fuzzy linguistic variables.

[0032] The rule base pre-sets multiple sets of logical reasoning rules to map membership combinations to the output variable space. The output variables include two types of judgment results: center model priority and edge model priority. The output reasoning mechanism calculates the activation intensity of each output variable based on the fuzzy reasoning results and converts the activation intensity of the output variable into a clear judgment signal through a defuzzification method. The prediction decision module selects to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model for subsequent temperature rise trend prediction based on the judgment signal.

[0033] In a preferred embodiment, the high-order multimodal temperature rise prediction model includes a temperature time-series data processing network, an image feature encoding network, and a modal fusion inference network. The temperature time-series data processing network is used to receive temperature evolution data of the target area over multiple historical time steps. The image feature encoding network is used to extract regional distribution features and edge temperature rise morphology information from the current infrared thermal imaging image. The modal fusion inference network fuses the output data of the temperature time-series data processing network and the image feature encoding network to construct a multimodal input tensor and input it into a deep neural network model for joint modeling and future prediction, outputting the temperature change trajectory of the target area over multiple future time steps.

[0034] The technical effects and advantages of this invention are as follows:

[0035] This invention transforms the temperature values ​​of thermal zones in continuous frame images acquired by an infrared thermal imaging device into temperature evolution data volumes with spatial positioning and temporal series attributes through an image conversion module. This constructs a structured spatiotemporal sequence foundation, providing high-quality, uniformly formatted data input for subsequent calculation modules. This approach not only enhances the ability to model the temporal continuity of temperature change processes but also improves the targeting of spatial analysis, enabling the system to accurately extract the temperature rise behavior of target areas from the image level, effectively supporting subsequent feature extraction and trend judgment.

[0036] This invention performs feature calculations on temperature evolution data through a fluctuation feature extraction module to construct a combination of fluctuation indicators, which is used to dynamically determine whether the temperature rise trend is stable. This enables fine-grained control over the timing of lightweight deep network model invocation in the edge prediction module. This design avoids using edge models with limited computing resources for prediction when there are drastic temperature fluctuations, improving the reliability of model prediction results and the overall system operating efficiency, while also reducing the computational load on edge devices under complex conditions.

[0037] This invention includes a prediction decision module. Based on the anomaly activation index and prediction offset index generated by the anomaly characterization module, it dynamically selects to call a high-order model deployed in a remote center or continue using the edge model to predict future temperature change trajectories, thus constructing an adaptive switching mechanism for prediction strategies. This mechanism enables on-demand allocation and execution of prediction paths, allowing the system to maintain high prediction accuracy and response efficiency under different thermal field change states, significantly improving the practicality and adaptability of the temperature rise trend early warning system. Attached Figure Description

[0038] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0039] Figure 1 This is a schematic diagram of the thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction in this invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0041] Reference Figure 1 The following examples were obtained:

[0042] Example 1:

[0043] A thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction includes an image conversion module, a fluctuation feature extraction module, an edge prediction module, an anomaly characterization module, and a prediction decision module.

[0044] The image conversion module transforms raw infrared thermal imaging data into structured prediction input data. Specifically, it first acquires continuous frame images of the target area using an infrared thermal imaging device, thereby capturing the real-time thermal distribution of the device, environment, or area. Then, based on the target area in the images, it extracts the spatial coordinates and corresponding timestamps from each frame, establishing a logical mapping between space and time through this coordinate-time correlation. Next, it extracts the thermal temperature values ​​within the target area from the images, organizes them into a temperature data sequence in chronological order, and combines this with the spatial coordinates to generate a dataset with spatial location identification and temporal evolution characteristics. Finally, the temperature evolution data output by this module constitutes a data structure that simultaneously possesses spatial positioning attributes and temporal series characteristics, serving as the spatiotemporal sequence structure required for subsequent processing and becoming the data foundation for the system's core prediction logic.

[0045] The fluctuation feature extraction module is responsible for extracting key features from the spatiotemporal sequence structure to determine whether the current temperature rise trend is stable. This module performs numerical modeling and statistical analysis on the temperature evolution data. First, it calculates the rate of temperature change per unit time by analyzing the mean temperature change at adjacent time points, thus obtaining the rate of temperature change, which measures the speed of thermal dynamic change. Second, using image analysis, it constructs a pixel window centered on the target region in each frame of the thermal image, extracts the gradient values ​​of all hotspot pixels within it, and calculates their root mean square values ​​to obtain the local temperature gradient intensity, representing the stability of the hotspot structure. These two numerical indicators together form a fluctuation index combination, used to determine the continuity and intensity of the thermal change process. When this index combination is compared with a preset stability reference interval in the system, it can be determined whether the current temperature rise state of the target region is in a stage of violent fluctuation or relatively stable operation.

[0046] The edge prediction module is the front-end prediction path in the system, its significance being to quickly and with low power consumption predict short-cycle temperature rise trends when the thermal field changes are relatively stable. This module achieves local prediction inference by calling a lightweight deep network model deployed at the thermal imaging acquisition end, offering advantages such as high real-time performance, fast response, and low resource consumption. The model takes the temperature evolution data volume output by the image conversion module as input, extracting the temperature value sequence of the target area at multiple consecutive time points. This sequence is then combined with several frames of original temperature data before the current time point and their first-order difference data to form a unified input tensor. After receiving this input, the model extracts temporal structure features and predicts the temperature trajectory for multiple future time steps based on a time-recursive approach. The prediction outputs of the previous n iterations are saved as a lightweight prediction curve for subsequent anomaly detection and stability assessment. The edge prediction module operates only when the volatility index combination determines that the current state is within a stable range; therefore, it is activated only when high-complexity calculations are not required.

[0047] The anomaly characterization module is a crucial component of the system, used to identify sudden surges in current heat distribution and to assess the stability of the prediction model. This module primarily generates two core indices for subsequent judgment: the anomaly activation index and the prediction offset index. The anomaly activation index obtains the degree of hotspot change in the current image frame through image gradient calculations and compares it with the average change trend in historical multi-frame images. Finally, it is mapped to a non-negative value through a normalization function to quantify whether the current hotspot activation level is higher than normal. The prediction offset index performs residual analysis on multiple lightweight prediction curves and their corresponding actual temperature rise trend data, combining window frequency and deviation magnitude to calculate the changing trend of model prediction reliability. The core function of this module is to promptly identify potential risks when hotspots surge or prediction deviations are severe, providing sufficient quantitative evidence for the decision-making module.

[0048] The prediction decision module is responsible for dynamically determining, based on the current thermal field state and prediction stability, whether to continue using the edge prediction path or switch to the higher-precision center prediction path during system operation. This module uses the anomaly activation index and prediction offset index output by the anomaly representation module as input variables, which are then fed into a trained machine learning model. In the current technical solution, this machine learning model is a fuzzy logic device. Through input membership transformation, rule base matching, and inference output processes, it maps the two index values ​​to discrete decision results, typically including either "continue edge prediction" or "switch to the center model." The decision result output by this module directly determines whether to invoke the high-order multimodal temperature rise prediction model of the remote center. Through this mechanism, the system achieves adaptive adjustment of the prediction strategy under different thermal field fluctuation states, improving overall accuracy and resource utilization efficiency.

[0049] The image conversion module is used to standardize and structure infrared thermal imaging image data. It is the first module in the entire early warning system to receive raw image information and convert it into a data structure that can be directly used by subsequent models. The image conversion module contains an image frame preprocessing structure, which consists of an image noise filtering component and a thermal zone identification component.

[0050] The image noise filtering component performs noise suppression processing on consecutive frames of images acquired by an infrared thermal imaging device. An infrared thermal imaging device is used to acquire the distribution of infrared radiation intensity on the surface of a target in a scene and outputs a sequence of thermal imaging image frames. Due to interference from the device itself or the imaging environment, the original thermal images often contain thermal noise such as background thermal disturbances, photosensitivity errors, and edge interference. Therefore, noise filtering processing of the image data is necessary. The image noise filtering component can employ edge-preserving median filtering, bilateral filtering, or thermal image intensity variance thresholding to remove high-frequency noise components without damaging the boundary features of the thermal spot structure, thereby improving image stability and subsequent recognition accuracy.

[0051] The thermal zone identification component automatically extracts the location boundaries and center reference points of target regions from images after noise suppression. The target region refers to a predefined or dynamically detected high-thermal-sensitive area in the image, typically representing a device or area to be monitored. The location boundary is the outer contour of the target determined by temperature thresholding, thermal density clustering, or image segmentation algorithms; the center reference point is the center pixel position of this region in the image coordinate system, which can be used for spatial localization and tracking. Based on the extraction results of the location boundaries and reference points, the thermal zone identification component generates spatial coordinate data of the target region in each frame of the image.

[0052] Spatial coordinate data represents the two-dimensional coordinate position of the target area within an image frame, including boundary range and center point position parameters. This spatial coordinate data is mapped one-to-one with the image frame's capture timestamp; the timestamp is the specific time information recorded by the thermal imaging device for acquiring each image frame. The spatial coordinate data and timestamps constitute an image frame sequence with positioning information, which serves as the standardized input data for the image conversion module before extracting temperature information.

[0053] After the image frame sequence is processed by the image conversion module, the temperature value of the hot zone in the target area of ​​each frame is extracted. The hot zone temperature value is the equivalent temperature value after converting the infrared radiation intensity of each pixel in the area. This extraction can be completed through a grayscale-temperature mapping model. Subsequently, the image conversion module binds the extracted hot zone temperature value of each frame image with spatial coordinate data and timestamp parameters to form a structured time-series temperature data set. This data set is a temperature evolution data volume with spatial positioning and time-series attributes. The temperature evolution data volume is a data structure that records the temperature change process of the target area at multiple consecutive time points and is stored in a structured form such as a table, tensor, or time-series array. The image conversion module finally outputs this temperature evolution data volume as the input basis for the subsequent fluctuation feature extraction module, which calculates the dynamic temperature change rate and local temperature gradient intensity, and further constructs a combination of fluctuation indicators.

[0054] The fluctuation feature extraction module performs structured numerical analysis on the temperature evolution data volume output by the image conversion module to quantify the dynamic characteristics of thermal changes in the current target region. The module's computational objective is to extract two numerical quantities to measure the rate of temperature change in the target region over time and the stability of the thermal gradient in its spatial structure, thus forming a combination of fluctuation indicators for state determination.

[0055] The temperature evolution data volume is a collection of data recording the temperature values ​​of the target region's thermal zone at multiple consecutive time points, including the correlation data between the target region's temperature value and the corresponding timestamp and spatial coordinates in each image frame. The fluctuation feature extraction module performs numerical calculations based on this data volume.

[0056] The temperature dynamic change rate reflects the overall temperature trend of a target area within a unit of time. It is calculated as follows: Temperature data from two adjacent time points are selected from the temperature evolution data volume. The mean value of all temperature pixels within the target area at each time point is calculated. Then, the difference between the two temperature means is used as the numerator, and the time interval between the two time points is used as the denominator. The resulting ratio is the current rate of temperature change per unit of time. This value reflects whether the target area is showing a trend of continuous warming, stabilization, or decline at the current stage; it is a direct quantitative result of the direction and magnitude of temperature change.

[0057] Local temperature gradient intensity reflects the spatial gradient distribution of hotspot structure in a target region within a single frame of an image. The specific calculation method is as follows: In each frame of the thermal image, a two-dimensional pixel window of a preset size is established, centered on the central reference point of the target region. This window limits the calculation range, ensuring that only the area within the target region is analyzed. Within the window, the grayscale values ​​of all hotspot pixels are extracted, and gradient calculations (such as based on the Sobel operator or gradient difference method) are performed on these pixels to obtain the grayscale changes of each pixel in the horizontal and vertical directions. Subsequently, all pixel gradient values ​​are squared, the average of the squared values ​​is calculated, and finally, the square root of this average is calculated to obtain the root mean square (RMS) value of the hotspot pixel gradient for that frame of the image. This value represents the local gradient intensity of the hotspot region in that frame of the image and is an important indicator for measuring the sharpness and concentration of hotspot edges.

[0058] The Sobel operator is a gradient operator commonly used in image processing to extract edge information. It is implemented based on the operation of local convolution kernels on the two-dimensional pixel matrix of the image. For a two-dimensional pixel window of the target region in a thermal image, convolution operations in two directions can be applied to each pixel position to extract the intensity of gray-level changes in the horizontal (x-direction) and vertical (y-direction) directions.

[0059] Specifically, the Sobel operator includes weighted convolution templates in two directions:

[0060] Horizontal template (Gx):

[0061] Vertical template (Gy):

[0062] Each target pixel forms a sliding window with its surrounding 3×3 pixels. This window is convolved with the two templates mentioned above to obtain the gradient values ​​Gx and Gy of that pixel in two directions. Taking the square root of the sum of the squares of these two values ​​gives the gradient magnitude G of that pixel. The G values ​​of all pixels are summed, and the average of their squared values ​​is calculated and then the square root is taken to obtain the local temperature gradient intensity value of the target region in the current frame image (i.e., the root mean square value of the hot spot pixel gradient).

[0063] The gradient difference method is a numerical approximation method used to calculate the local variation trend of pixel gray values ​​in different directions in an image. In a two-dimensional image coordinate system, the gray-level gradient of each pixel in the target region is approximately solved in both the row and column directions. For each pixel I(i,j), its gradient in the x-direction (row direction) and y-direction (column direction) can be calculated using the following difference methods:

[0064] Approximate gradient value in the x-direction: Gx = I(i+1,j) - I(i-1,j);

[0065] Approximate gradient value in the y-direction: Gy = I(i,j+1) - I(i,j-1);

[0066] This method does not require a convolution kernel, making it suitable for fast gradient estimation with low computational cost in edge computing environments. After obtaining Gx and Gy, the same approach is used... Obtain the gradient magnitude G of the pixel. For a complete two-dimensional pixel window, sum the squares of the G values ​​of all pixels, take the mean, and take the square root to obtain the root mean square value of the gradient of the hot spot region in the frame image, which is used as the local temperature gradient intensity of the frame.

[0067] The Sobel operator offers high accuracy and more precise edge response, making it suitable for center analysis models with high requirements for hot spot edge localization. The gradient difference method has low computational complexity and is suitable for real-time processing, allowing it to be deployed in edge prediction modules for rapid response.

[0068] The two values ​​described above constitute a volatility index combination, named Data 1 and Data 2, respectively. Data 1 is obtained by selecting a preset fixed time window (e.g., m consecutive frames of images), and calculating the arithmetic mean of the temperature change rates for each pair of adjacent time points within this window; this average value is Data 1. Data 2 is obtained by calculating the root mean square value of the local temperature gradient in each frame of the same time window, and selecting the maximum value as Data 2. This volatility index combination represents the overall trend and local structural stability of the current target region in terms of thermal behavior.

[0069] Determining whether the temperature rise trend is stable involves comparing the combination of volatility indicators with a pre-defined range of stability reference indicators within the system. This reference range defines a dual-threshold range to determine under what conditions the system can classify the temperature rise in the target area as stable. If the values ​​of both data points 1 and 2 in the volatility indicator combination fall within the boundaries of their respective reference ranges, the current temperature rise trend is determined to be stable; conversely, if either indicator exceeds the reference range boundary, the current temperature rise trend is determined to be unstable. The output of the volatility feature extraction module is a quantitative judgment signal of the current thermal change state, which directly affects the activation decision of subsequent edge prediction or anomaly characterization modules.

[0070] The edge prediction module is a functional module in the system that performs local rapid prediction tasks when the thermal change trend of the target area is relatively stable. Its core component is a lightweight deep network model deployed at the thermal imaging acquisition end. This model is activated when the combination of volatility indicators is determined to be in a stable state, and is used to predict the temperature change trend of the target area over the next few time steps, meeting the requirements of low latency and low computational overhead. The lightweight deep network model deployed in the edge prediction module refers to a time series modeling neural network model with a simplified structure, low parameter count, and adaptability to the operating conditions of edge computing devices. It can be built based on existing technologies such as Temporal Convolutional Networks (TCN) or Long Short-Term Memory Networks (LSTM).

[0071] Temporal convolutional networks are neural network structures that utilize one-dimensional convolutions to slide along the temporal dimension. By stacking multiple convolutional layers, they can extract long-term and short-term dependencies in an input sequence. Their core advantages lie in their ability to process input sequences in parallel, their fixed receptive field, and their capacity to capture local to global temporal dependency patterns.

[0072] Long Short-Term Memory (LSTM) networks are a variant of Recurrent Neural Networks (RNNs) with internal gate control mechanisms such as input gates, forget gates, and output gates. They are able to effectively learn long-term dependencies in time series and perform well in sequence prediction tasks with time delays or historical trends.

[0073] The lightweight deep network model maintains the basic structural features of temporal convolutional networks or long short-term memory networks, and is customized by adopting a lightweight compression strategy, differential fusion input structure, and edge computing adaptation. The number of model parameters is controlled within a preset upper limit, and the predicted temperature change results of the target area in the future multiple time steps are output through a recursive method.

[0074] Lightweight Compression Strategy: This strategy includes channel pruning, layer pruning, low-order weight quantization, weight sharing, and sparsification. Channel pruning evaluates the importance of output channels in convolutional layers or LSTM units, retaining main channels and eliminating redundant connections; layer pruning limits the model depth to 1 to 2 layers; weights are converted to 8-bit fixed-point representations and run in the quantization inference engine of the edge chip. Simultaneously, sparse connections can be performed on the weight matrices between temporal units to compress the model size to a predetermined parameter limit (e.g., less than 500K).

[0075] Differential fusion input structure: The input data source for the lightweight deep network model is the temperature evolution data volume output by the image conversion module. The temperature evolution data volume is a data set that records the temperature changes of the target region's hot zone at multiple time points, with time as the main axis. It includes the hot zone temperature value extracted from each frame of the image, spatial location data, and the corresponding timestamp.

[0076] When the edge prediction module calls the lightweight deep network model, it extracts the temperature value sequence of the target region over multiple consecutive time steps from the temperature evolution data volume, using it as the model's input data. This input data consists of two parts: first, the raw thermal imaging temperature data extracted from several frames of images prior to the current time point; and second, the first-order differential temperature change data composed of the temperature differences between adjacent frames.

[0077] First-order differences represent the difference between the temperature value at any given time point and the temperature value at the previous time point. This is a fundamental feature construction method in time series analysis, helping to improve the model's sensitivity to dynamic features such as fluctuation trends and heating rates. The original sequence and the difference sequence can be combined into a unified input tensor through channel-dimensional concatenation for subsequent modeling.

[0078] Edge computing adaptation: The model is deployed and runs at the edge, adapting to the AI ​​chips, embedded GPUs, or NPU modules integrated into thermal imaging acquisition devices. The deployment process includes steps such as model structure conversion, quantization inference format conversion, and input tensor size standardization. The model is loaded through inference engines such as TFLite, ONNX, or TensorRT, and performs temperature sequence prediction in a single-sample stream manner to avoid high memory consumption. During operation on the edge platform, the model is only invoked when the temperature rise trend is determined to be stable, further reducing energy consumption and load, and achieving low-power prediction closed-loop control at the edge of the thermal imaging system.

[0079] Recursive prediction mechanism: After receiving the constructed input tensor, the model performs feature extraction operations on the time series structure, including convolutional feature extraction, gating mechanism activation, or time-weighted aggregation as used in existing technologies, to extract trend changes, periodic fluctuations, or local anomalous signals in the sequence. These will not be elaborated upon here. After feature extraction, the edge prediction module outputs a sequence of predicted temperature values ​​for the target region over multiple future time steps using a recursive approach.

[0080] The recursive approach refers to a model that outputs only the predicted value for the next time step at a time, using this prediction as part of a new input sequence to progressively update the model input. This process generates multi-step temperature trajectories without increasing the model size. The final output serves as a temperature rise prediction curve for the current target area over a future period. This curve can be used by the anomaly characterization module for calculating the prediction offset index or directly for front-end alarm visualization and early warning strategy linkage. In the edge prediction module, a lightweight deep network model uses the recursive approach to output temperature predictions for multiple future time steps. The recursive approach is a sequence generation strategy that feeds back the model's own prediction results and incorporates them into subsequent prediction processes. It addresses the technical challenge of a model structure that only supports single-step output when the task objective is multi-step prediction. The processing flow includes the following steps: At the current time point t0, the last few frames of temperature data and the corresponding first-order difference data in the input sequence are input into the model, and the predicted temperature value for the first future time step t1 is output; the prediction result at time t1 is used as the input of the new frame to update the input sequence, forming the next input tensor; the updated tensor is used for model inference again to generate the prediction result for the next time step t2; the above operation is repeated until all predicted values ​​within the future target time range are obtained. The core advantage of this method is that it can adapt to multi-step prediction tasks of arbitrary length without changing the model structure. However, to ensure prediction accuracy, the number of prediction steps is usually limited to a certain range, and the stability of the input structure can be maintained by using a sliding window method.

[0081] The anomalous activation index is used to reflect whether the hotspot gradient of the target region in the current image frame deviates drastically from the historical trend. Its calculation is based on the hotspot change structure in the current image and historical images, and specifically includes the following steps:

[0082] Current Hotspot Gradient Change Extraction: A one-dimensional or two-dimensional Sobel operator operation is performed on the temperature distribution of the target region in the current infrared thermal imaging image. The Sobel operator is an edge extraction algorithm in image processing that extracts the intensity of change in grayscale or temperature values ​​in a certain direction (horizontal or vertical) through convolution operations. If a one-dimensional Sobel operator is used, unidirectional gradient estimation is performed on the image in either the horizontal or vertical direction. If a two-dimensional Sobel operator is used, gradient components in both directions are obtained simultaneously, and the gradient magnitude of each pixel is calculated using the square root of the sum of squares. The appropriate dimension can be selected according to the required computational accuracy. After performing the Sobel operation on all pixels in the target region, the temperature change value of each pixel in the gradient direction can be obtained. Subsequently, a local search is performed within the target region based on a sliding window of fixed size. The sliding window is a sub-region that slides across the image pixel matrix with a fixed width and height for local computation. The gradient magnitude of all pixels within the window is extracted, and the local maximum gradient change value is extracted within each window. After traversing the entire target region, the largest local maximum value is selected and defined as the absolute value of the maximum hotspot gradient change in the current image frame. This value measures the point where the hotspot edge changes most drastically in the current frame and is a key indicator for measuring hotspot activity.

[0083] Historical average change extraction: From the target region in the previous m frames, extract the absolute value of the maximum hotspot gradient change in each frame, calculated in the same way as the current image. These values ​​form a numerical sequence of length m. Perform an arithmetic mean operation on this numerical sequence, that is, sum all the historical maximum values ​​and divide by the number of frames m to obtain an average value, which is defined as the historical average change. This historical average change reflects the baseline of hotspot activity in the target region over a short period of time and serves as a benchmark for comparing whether the current frame is abnormal.

[0084] Normalized Deviation Calculation: The absolute value of the maximum hotspot gradient change calculated in the current image frame is subtracted from the historical average change. The difference between the two is defined as the deviation. This deviation reflects the magnitude of the deviation between the current hotspot intensity and the historical average. To enhance the stability of this deviation, a normalization operation is then performed. The sum of the historical average change and a preset stability constant is used as the denominator. This stability constant is a non-zero positive number, much smaller than the temperature gradient magnitude, to prevent mathematical errors such as division by zero when the historical average change is too small or even zero. The normalized result represents the standardized difference between the current hotspot activity and the historical baseline.

[0085] Nonnegation processing and determination of the anomalous activation index: The normalized deviation result is compared with zero. If the normalized value is less than zero, it indicates that the current change is lower than the historical baseline and does not constitute an anomaly, so it is taken as zero; if the value is greater than zero, it indicates that there is a significant surge in the current image frame, so the value is retained. The final result is defined as the anomalous activation index, which is a non-negative real number and is positively correlated with the degree of hotspot anomaly in the current image.

[0086] The prediction offset index is used to evaluate the stability and deviation of prediction results generated by a lightweight deep network model over multiple consecutive calls. It serves as a dynamic monitoring metric for the model's prediction performance. The calculation of this index relies on the prediction data generated by the edge prediction module at multiple time points and the corresponding real observation data. The overall calculation process includes the following three stages:

[0087] We acquire n lightweight prediction curves generated by the lightweight deep network model during the first n calls. Each prediction curve represents the predicted temperature change sequence of the target area over multiple time steps based on the input data at that time. Simultaneously, we acquire the actual temperature rise trend data corresponding to these n calls, i.e., the actual temperature change sequence of the target area within the same time range after the prediction occurs. For each prediction curve and its corresponding actual temperature rise trend data, we pair them one-to-one by time step, calculate the difference between the predicted temperature value and the actual temperature value at the same time point, and square this difference to represent the residual squared value at that time point. We sum the residual squared values ​​for all time steps to obtain the residual sum of squares corresponding to that prediction curve. We repeat this process to obtain n residual sums of squares, forming a residual sequence of length n. This sequence reflects the cumulative prediction error of the edge model over multiple time steps.

[0088] The residual sequence is divided into equal-width sliding windows, meaning the n sums of squared residuals are divided into several continuous subsequence windows of equal length, each containing a fixed number of residual values. Within each sliding window, the maximum squared residual value is extracted as the local maximum deviation for that window. The frequency of each squared residual value appearing as the maximum across all windows is calculated. This frequency is divided by the total number of windows to obtain the local surge ratio corresponding to each maximum squared residual value. This ratio reflects the concentration of anomalous deviations within certain prediction time periods. The maximum squared residual value within each sliding window is multiplied by its corresponding local surge ratio, and then all weighted residual values ​​are summed and normalized by the total number of sliding windows. The result is defined as the residual confidence weight. This weight is used to comprehensively evaluate the severity and frequency of locally high deviation intervals in multiple predictions by the model.

[0089] Statistical processing is performed on the n squared residual values ​​constituting the residual sequence, calculating the standard deviation (representing the volatility of the residual values) and range (the difference between the maximum and minimum values, reflecting the maximum dispersion) of the sequence. The standard deviation and range are summed to define a prediction stability factor, which represents the consistency of the model's prediction accuracy across different time periods. Finally, the residual confidence weights are multiplied by the prediction stability factor to obtain the prediction bias index. This index is a non-negative real number; a larger value indicates poorer model prediction stability and more severe overall deviation. It is useful for determining whether to switch to a remote center model for high-precision prediction or trigger a model retraining mechanism.

[0090] Taking n=10 as an example, assuming the sum of squared residuals from 10 predictions is [1.2, 0.9, 1.5, 3.8, 0.7, 2.2, 1.0, 0.8, 4.1, 1.3], divided into 5 windows, each containing 2 values. The maximum value selected from each window is [1.2, 1.5, 3.8, 2.2, 4.1]. The frequency of each value being the maximum value in the sliding window is calculated, assuming it's [1, 1, 1, 1, 1]. The surge ratio is the mean 1 / 5. The weighted total is: (1.2 + 1.5 + 3.8 + 2.2 + 4.1) × 1 / 5 = 12.8 × 0.2 = 2.56. Simultaneously, the standard deviation of the original residual sequence is set to 1.02, the range to 4.1 - 0.7 = 3.4, and the prediction stability factor to 1.02 + 3.4 = 4.42. The final predicted offset index is 2.56 × 4.42 ≈ 11.32, indicating that there is a significant inconsistency in the prediction results of the model across multiple calls.

[0091] The predictive decision module is used during system operation to determine whether to switch model paths based on the dynamic state of thermal rise in the current target area and the predicted performance of the edge model. The pre-trained machine learning model used in this module is a fuzzy logic device. A fuzzy logic device is a logical reasoning structure built on fuzzy set theory, capable of handling the mapping between continuous value inputs and linguistic rules, and suitable for industrial decision-making scenarios involving uncertainty and fuzzy boundary judgments.

[0092] The fuzzy logic unit includes an input membership function set, whose main function is to receive two core input parameters: the anomalous activation index (reflecting the magnitude of hotspot surges in the target region) and the prediction offset index (reflecting the stability of edge model predictions). The input membership function set defines several membership functions for each input parameter, typically trigonometric, Gaussian, or trapezoidal functions, to map real-valued inputs to membership degrees. Each input parameter is divided into several fuzzy linguistic variable intervals, such as "low," "medium," and "high." When the value of the anomalous activation index or prediction offset index falls into the corresponding interval, it will be assigned to that fuzzy linguistic variable with a certain membership degree. For example, an anomalous activation index of 0.45 may simultaneously have membership degrees of "medium (0.7)" and "high (0.3)."

[0093] The fuzzy logic generator has a predefined rule base containing several "if-then" logic rules based on fuzzy language. Each rule is defined with the following structure: If the abnormal activation index is [Level A] and the prediction offset index is [Level B]; Then the output decision is [Model Selection]. Here, [Level A] and [Level B] correspond to the fuzzy language variables defined by the input membership functions mentioned above. [Model Selection] selects one of the two output variables: Edge model priority: indicates that the system continues to use the lightweight deep network model currently deployed at the thermal imaging acquisition end for prediction; Center model priority: indicates that the system needs to switch to a remote center to call a higher-order multimodal temperature rise prediction model to improve prediction accuracy. For example, the rule can be set as: If the abnormal activation index is "high" and the prediction offset index is "high" Then center model priority; or If the abnormal activation index is "low" and the prediction offset index is "medium" Then edge model priority. The rule base generally contains complete rule groups covering all possible input combinations, such as 3×3 rules (9 rules in total).

[0094] After receiving the input parameters and calculating the membership degrees, the fuzzy logic unit performs fuzzy inference on all matching rules based on the rule base. The fuzzy inference process involves calculating the activation strength of the output variable corresponding to each rule, typically using a "min-implication" or "multiplicative normalization" method. For example, if the input belongs to "medium (0.6)" and "high (0.5)", the activation strength of the corresponding rule is 0.6 × 0.5 = 0.3. The activation results of all rules that produce "edge model priority" and "center model priority" are merged to obtain the overall activation map of the two output variables.

[0095] After the fuzzy inference results are generated, the system converts the activation intensity into a deterministic numerical or categorical output using defuzzification methods. Commonly used defuzzification methods include: maximum membership degree method: selecting the output with the highest membership degree; centroid method: calculating the centroid position of the fuzzy output membership graph. In this system, for rapid decision-making, the maximum membership degree method is used as the defuzzification mechanism, directly selecting the output variable with the highest activation intensity as the final decision signal.

[0096] Based on the deblurring results, i.e., the explicit determination signal, the prediction decision module selects the model invocation path: if the determination result is "central model priority," the system automatically calls the high-order multimodal temperature rise prediction model deployed on the remote central platform; if the determination result is "edge model priority," the lightweight deep network model deployed at the thermal imaging acquisition end continues to be used, maintaining the current prediction strategy. This mechanism constitutes the key decision logic for the system to achieve dynamic switching and optimized response of prediction models under real-time monitoring.

[0097] The high-order multimodal temperature rise prediction model includes a temperature time series data processing network, an image feature encoding network, and a modal fusion inference network. The temperature time series data processing network is used to receive temperature evolution data of the target area over multiple historical time steps. The image feature encoding network is used to extract regional distribution features and edge temperature rise morphology information from the current infrared thermal imaging image. The modal fusion inference network fuses the output data of the temperature time series data processing network and the image feature encoding network to construct a multimodal input tensor and input it into a deep neural network model for joint modeling and future prediction, outputting the temperature change trajectory of the target area over multiple future time steps.

[0098] The high-order multimodal temperature rise prediction model is the core prediction module deployed on the remote central computing platform in the system. It possesses multi-source data fusion processing capabilities and deep modeling capabilities, used to provide higher-accuracy thermal imaging temperature rise trend prediction results when edge prediction strategies are insufficient. Specifically, it includes:

[0099] Temperature Time Series Data Processing Network: This network receives temperature evolution data volumes of a target area over multiple historical time steps. The temperature evolution data volume is generated by the image conversion module and is a data structure with spatial positioning and temporal series attributes. It includes the thermal zone temperature value of the target area in each frame of infrared thermal imaging, the corresponding spatial coordinate data, and the capture timestamp.

[0100] This network is built upon a time-series neural network, and a typical structure includes:

[0101] Multi-layer one-dimensional convolutional stacked structure; or multi-gated recurrent network structure; or Transformer encoder structure, used for self-attention extraction of sequence features.

[0102] The network performs deep encoding on the temperature evolution trend over time, extracts long-term dependent features, periodic change features, and local temperature anomaly response patterns, and outputs an embedding vector representing the historical temperature rise behavior of the target area.

[0103] Image Feature Coding Network: The image feature coding network is used to extract regional distribution features and edge temperature rise morphology information from the current infrared thermal imaging image. The input is the latest frame of infrared image acquired by the system, whose image content includes the hot spot morphology, intensity distribution, and differences from the surrounding background of the target area.

[0104] This network is a typical two-dimensional image coding neural network, usually employing:

[0105] Convolutional neural network structures (such as ResNet, MobileNet, etc.) are used; edge-preserving modules and spatial attention mechanisms are added to enhance thermal boundary detection; existing techniques will not be elaborated here. The output channel is a feature map or region feature vector containing spatial structure information. The purpose of the network is to extract the spatial distribution features, edge contour abrupt change features, and temperature gradient clustering features of the target region in the current frame image, forming a structured representation of the static visual dimension.

[0106] Modal fusion inference network: The modal fusion inference network is the core module of the high-order multimodal temperature rise prediction model. Its function is to effectively fuse the output data of the temperature time series data processing network and the image feature encoding network to construct a unified input representation, which is then input into the deep neural network for joint modeling and future prediction. Fusion methods include, but are not limited to:

[0107] The method employs vector concatenation, additive fusion, and multiplicative attention mechanisms. A cross-attention mechanism is used to adjust the weights of features from both modalities; existing techniques are not detailed here. The final output is a unified multimodal input tensor, with a dimensional structure adapted for subsequent prediction modeling modules. This fused multimodal input tensor is fed into a deep prediction model, which can be a multi-layer feedforward neural network, a temporal modeling network with residual connections, or an encoder-decoder network. By jointly modeling the time-series trend and image spatial patterns, the model predicts the temperature change trajectory of the target region over multiple future time steps.

[0108] The final output is a temperature prediction sequence, representing the temperature evolution trend of the target area over several future time steps. The output structure is a one-dimensional time series or a two-dimensional temperature field tensor, which can be used as input for subsequent decision-making mechanisms to determine risk level, compare early warning thresholds, or identify thermal runaway trends.

[0109] For example, using a temperature evolution data volume composed of nearly 10 frames of heatmaps of the target area as the time-series input, and simultaneously acquiring the current frame heatmap as the image input, these are fed into their respective processing networks, resulting in a 64-dimensional time vector and a 128-dimensional spatial feature vector. The modal fusion inference network uses attention-weighted concatenation to construct a fusion tensor, which is then fed into a multi-step regression prediction network with a residual structure, outputting the predicted temperature values ​​for the next 5 time points: [57.3, 58.9, 59.2, 60.1, 61.0], in °C. This result is used to determine whether the temperature exceeds the warning threshold (e.g., 60 °C) set in subsequent operations. If it does, the system can trigger an abnormal state report.

[0110] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0111] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0112] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0114] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction, characterized in that, It includes an image conversion module, a fluctuation feature extraction module, an edge prediction module, an anomaly representation module, and a prediction decision module; The image conversion module acquires continuous frame images through an infrared thermal imaging device. Based on the spatial coordinates of the target area in the image and the shooting timestamp, it extracts the thermal zone temperature value in each frame and constructs a temperature evolution data volume with spatial positioning and time series attributes, forming a spatiotemporal sequence structure. The fluctuation feature extraction module calculates the rate of temperature change per unit time and the root mean square of the gradient of hot spot pixels in the thermal image for the spatiotemporal sequence structure. It obtains the dynamic rate of temperature change and the intensity of local temperature gradient, and forms a combination of fluctuation indicators to determine whether the temperature rise trend is in a stable state. The edge prediction module, when the combination of volatility indicators is in a stable range, calls a lightweight deep network model deployed at the thermal imaging acquisition end to predict the temperature changes of the target area in multiple future time steps and use it as a basis for early warning. The first n calls to the lightweight deep network model each output a prediction curve, and the n prediction curves output by these n calls together constitute the lightweight prediction curve, which is the set of these n prediction curves. The anomaly characterization module generates an anomaly activation index based on the difference between the absolute value of the maximum hotspot gradient change in the target area and the historical average change when the combination of volatility indicators exceeds the stable range. Furthermore, it generates a prediction offset index based on the mean square residual between the lightweight prediction curve and the actual temperature rise trend data corresponding to the previous n calls to the lightweight deep network model, in order to evaluate the prediction stability of the edge model. The prediction and decision module receives the abnormal activation index and prediction offset index as inputs, which are then fed into the trained machine learning model. Based on the judgment result of the machine learning model, it selects to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model, and outputs the temperature change trajectory of the target area in the future multiple time steps as the basis for early warning.

2. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 1, characterized in that, The image conversion module has an image frame preprocessing structure consisting of an image noise filtering component and a thermal zone recognition component. The image noise filtering component is used to suppress noise in the continuous frame images acquired by the infrared thermal imaging device. The thermal zone recognition component is used to automatically extract the position boundary and center reference point of the target area in the image and generate spatial coordinate data. The spatial coordinate data is mapped one-to-one with the shooting timestamp to form an image frame sequence with positioning information. After the image frame sequence is processed by the image conversion module, the temperature value of the hot zone in each frame is extracted. Combined with spatial coordinate data and shooting timestamp parameters, a temperature evolution data volume with spatial positioning and time series attributes is generated. The temperature evolution data volume stores the temperature change information of the target area at multiple consecutive time points in a structured manner. The image conversion module finally outputs the temperature evolution data volume, which serves as the input basis for the subsequent fluctuation feature extraction module to calculate the combination of fluctuation indicators.

3. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 2, characterized in that, The fluctuation feature extraction module performs numerical calculations based on the temperature evolution data volume in the spatiotemporal sequence structure. The temperature dynamic change rate is obtained by dividing the difference between the mean temperature of the target area at two adjacent time points in the temperature evolution data volume by the time interval, forming a numerical expression reflecting the rate of temperature change per unit time. The local temperature gradient intensity is obtained by establishing a two-dimensional pixel window centered on the target region in each frame of the thermal image, extracting the gray value gradient of all hot spot pixels within the two-dimensional pixel window, and then taking the square root of the average of the sum of squares of all pixel gradient values ​​to obtain the root mean square value of the hot spot pixel gradient of the corresponding frame.

4. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 3, characterized in that, The volatility indicator portfolio consists of the following data sets: Data Set 1 and Data Set 2. Data 1 was obtained by averaging multiple temperature change rates within a preset fixed time window; Data 1 was obtained by finding the root mean square maximum of the local temperature gradient in multiple frames of images within a preset fixed time window; Determining whether the temperature rise trend is in a stable state refers to: The volatility index combination is compared with the preset stability reference index combination's value range. If the volatility index combination falls within the preset stability reference index combination's value range, the temperature rise trend is in a stable state; otherwise, the temperature rise trend is not in a stable state.

5. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 4, characterized in that, The lightweight deep network model deployed in the edge prediction module is built based on a temporal convolutional network or a long short-term memory network. The input data comes from the temperature evolution data volume output by the image conversion module. When the edge prediction module calls the lightweight deep network model, it uses the temperature value sequence of the target area in multiple consecutive time steps as the input data. The temperature value sequence includes the original thermal imaging temperature data and the first-order differential temperature change data of several frames before the current time point. After receiving input data, the lightweight deep network model performs pre-defined feature extraction on the time series structure and generates a series of predicted temperature values ​​for multiple future time steps in the target area through a recursive approach.

6. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 5, characterized in that, The abnormal activation index is generated in the following way: Perform one-dimensional or two-dimensional Sobel operator operations on the temperature distribution of the target area in the current infrared thermal imaging image, calculate the temperature change value of each pixel in the gradient direction, and extract all local maximum gradient change values ​​in the target area based on a sliding window of fixed size, and select the global maximum value as the absolute value of the maximum hot spot gradient change in the current image frame. Extract the absolute value of the maximum hot spot gradient change of the target region from the first m frames of images, and perform an arithmetic average to obtain the historical average change as a reference metric. The deviation is obtained by subtracting the historical average change from the absolute value of the maximum hotspot gradient change in the current image frame. Then, the deviation is normalized by using the sum of the historical average change and the preset stability constant as the denominator. The stability constant is a non-zero positive number to avoid numerical instability caused by the denominator approaching zero. Finally, the normalized result is compared with zero. If the result is less than zero, it is taken as zero. If the result is greater than zero, its original value is retained. This determines the abnormal activation index.

7. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 6, characterized in that, The predicted offset index is generated in the following way: Obtain n lightweight prediction curves generated by the lightweight deep network model during the first n calls, and simultaneously obtain the actual temperature rise trend data corresponding to each call. Calculate the squared difference between the predicted value and the actual value for each prediction curve and its corresponding actual temperature rise trend data at multiple time steps to obtain n residual sums of squares, forming a residual sequence. The residual sequence is divided into equal-width sliding windows. The maximum squared residual value is extracted in each window, and its frequency of occurrence in all windows is counted. The ratio of this frequency value to the total number of windows is defined as the local surge ratio. The maximum squared residual value of each window is multiplied by its corresponding local surge ratio, the sum is obtained, and then divided by the total number of windows to obtain the residual confidence weight. Calculate the standard deviation of the squared residuals for n residuals, and calculate the difference between the maximum and minimum values. The sum of the standard deviation and the range is used as the prediction stability factor. Multiply the residual confidence weights by the prediction stability factor to obtain the prediction bias index.

8. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 7, characterized in that, The machine learning model used in the prediction decision module is a fuzzy logic device. The fuzzy logic device includes an input membership function set, a rule base and an output inference mechanism. The input membership function set is used to receive the abnormal activation index and the prediction offset index as input parameters, and converts the two input parameters into corresponding membership values ​​based on the set fuzzy linguistic variables. The rule base pre-sets multiple sets of logical reasoning rules to map membership combinations to the output variable space, where the output variables include two types of judgment results: center model priority and edge model priority. The output inference mechanism calculates the activation intensity of each output variable based on the fuzzy inference results, and converts the activation intensity of the output variable into a clear decision signal through a defuzzification method. The prediction decision module selects to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model for subsequent temperature rise trend prediction based on the decision signal.

9. The thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction according to claim 8, characterized in that, The high-order multimodal temperature rise prediction model includes a temperature time-series data processing network, an image feature encoding network, and a modal fusion inference network. The temperature time-series data processing network is used to receive temperature evolution data of the target area over multiple historical time steps. The image feature encoding network is used to extract regional distribution features and edge temperature rise morphology information from the current infrared thermal imaging image. The modal fusion inference network fuses the output data of the temperature time-series data processing network and the image feature encoding network to construct a multimodal input tensor and input it into a deep neural network model for joint modeling and future prediction, outputting the temperature change trajectory of the target area over multiple future time steps.

Images