Thermal imaging temperature rise trend early warning system based on space-time sequence prediction

Abstract

The invention discloses a thermal imaging temperature rise trend early warning system based on time-space sequence prediction, and particularly relates to the technical field of thermal imaging data prediction and early warning. The thermal imaging temperature rise trend early warning system comprises an image conversion module, a fluctuation feature extraction module, an edge prediction module, an anomaly characterization module and a prediction decision module; a temperature dynamic change rate and gradient intensity are calculated, edge model prediction is carried out based on a fluctuation index combination, when the fluctuation index combination does not exceed a stable interval, a lightweight deep network model deployed at a thermal imaging acquisition end is called, and when the fluctuation index combination exceeds the stable interval, a prediction decision module determines whether to switch to a high-order multi-modal model; the space-time information extraction capability is improved by constructing the temperature evolution data body, the prediction path is dynamically controlled based on the fluctuation index combination, and the prediction stability and efficiency are improved; and the abnormal activation index and the prediction offset index are combined to realize adaptive switching of model calling, so that the accuracy and adaptability of the early warning system are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of thermal imaging data prediction and early warning, and more specifically, to a thermal imaging temperature rise trend early warning system based on spatiotemporal sequence prediction. Background Art

[0002] With the rapid development of IoT technology and intelligent inspection equipment, infrared thermal imaging, as a non-contact, efficient detection method, has been widely used for operational status monitoring tasks in a variety of key scenarios, including power systems, industrial station control areas, smart substations, rail transit, and energy and chemical industries. By continuously collecting infrared radiation images of the target equipment surface, this type of equipment can accurately reflect its temperature distribution without interfering with the equipment's operation, thereby enabling early identification of internal operating loads, thermal stress accumulation, and potential failure risks. Thermal imaging monitoring, particularly in scenarios with high voltage, high power, or complex electrical structures, is gaining increasing attention as a vital part of ensuring operational safety.

[0003] However, existing infrared thermal imaging monitoring systems still face numerous technical bottlenecks during deployment. Most systems still rely primarily on static analysis of single-frame thermal images or simple threshold determination of current temperature values. These systems struggle to fully exploit the temporal evolution patterns inherent in multiple consecutive frames, and their ability to dynamically predict temperature trends is severely limited. These methods often only generate alarms after temperatures exceed specified limits or hot spots accumulate. This makes it difficult to promptly indicate the impending signs of an abnormal device state, leading to the risk of delayed response and ineffective early warnings.

[0004] In addition, although some systems have introduced prediction models based on machine learning, their deployment environments are mostly embedded edge computing devices, which have limited computing power, memory, and reasoning capabilities. This results in the need to significantly tailor the model structure, making it impossible to balance the stability and accuracy of the prediction. In particular, when faced with complex working conditions such as drastic fluctuations in the thermal field and sudden changes in equipment load, the prediction results are prone to significant deviations. At the same time, current systems usually adopt a fixed model strategy and do not have the ability to flexibly adjust the model structure or path according to the operating status. They lack an intelligent discrimination mechanism at the model call level, resulting in a slow response when switching between thermally stable and unstable states. The overall system exhibits obvious rigidity, passivity, and lack of adaptability. Therefore, the present invention proposes a thermal imaging temperature rise trend warning system based on spatiotemporal series prediction in order to solve the above problems. Summary of the Invention

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] Thermal imaging temperature rise trend warning system based on spatiotemporal sequence prediction, including image conversion module, fluctuation feature extraction module, edge prediction module, anomaly characterization module and prediction decision module;

[0007] The image conversion module acquires continuous frame images through infrared thermal imaging equipment, extracts the temperature value of the hot zone in each frame based on the spatial coordinates of the target area in the image and the shooting timestamp, constructs a temperature evolution data volume with spatial positioning and time series attributes, and forms a spatiotemporal sequence structure;

[0008] The fluctuation feature extraction module calculates the temperature change rate per unit time and the root mean square of the hot spot pixel gradient in the thermal image based on the spatiotemporal sequence structure, respectively, to obtain the temperature dynamic change rate and local temperature gradient intensity, forming a combination of fluctuation indicators to determine whether the temperature rise trend is in a stable state;

[0009] When the volatility indicator combination is in a stable range, the edge prediction module calls the lightweight deep network model deployed on the thermal imaging acquisition end to predict the temperature changes of the target area in multiple time steps in the future and use it as a basis for early warning. The prediction curves output by the first n calls of the lightweight deep network model constitute the lightweight prediction curve;

[0010] The anomaly characterization module generates an anomaly activation index based on the difference between the absolute value of the maximum hot spot gradient change and the historical average change in the target area when the volatility indicator combination exceeds the stable range. It also generates a prediction offset index based on the mean square residual between the lightweight prediction curve and the actual temperature rise trend data corresponding to the previous n lightweight deep network model calls to evaluate the prediction stability of the edge model.

[0011] The prediction decision module receives the abnormal activation index and the prediction offset index as input, and inputs them into the trained machine learning model. Based on the judgment results of the machine learning model, it chooses to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model, and outputs the temperature change trajectory of the target area in multiple time steps in the future as an early warning basis.

[0012] In a preferred embodiment, the image conversion module has an image frame preprocessing structure consisting of an image noise filtering component and a hotspot recognition component. The image noise filtering component is used to suppress noise in continuous frame images acquired by the infrared thermal imaging device. The hotspot recognition component is used to automatically extract the position boundary and center reference point of the target area in the image and generate spatial coordinate data. The spatial coordinate data is mapped one-to-one with the shooting timestamp to form an image frame sequence with positioning information.

[0013] After the image frame sequence is processed by the image conversion module, the temperature value of the hot zone in each frame is extracted and combined with the spatial coordinate data and the shooting timestamp parameters to generate a temperature evolution data volume with spatial positioning and time series attributes. The temperature evolution data volume stores the temperature change information of the target area at multiple consecutive time points in a structured manner. The image conversion module finally outputs the temperature evolution data volume as the input basis for the subsequent fluctuation feature extraction module to calculate the volatility indicator combination.

[0014] In a preferred embodiment, the fluctuation feature extraction module performs numerical calculation operations based on the temperature evolution data body in the spatiotemporal sequence structure, and the temperature dynamic change rate is obtained by dividing the difference between the mean temperature of the target area at two adjacent time points in the temperature evolution data body by the time interval value to form a numerical expression reflecting the temperature change rate per unit time;

[0015] The local temperature gradient intensity is obtained by establishing a two-dimensional pixel window with the target area as the center in each frame of the thermal image, extracting the grayscale value gradient of all hot spot pixels in the two-dimensional pixel window, and calculating the average of the sum of the squares of all pixel gradient values ​​and then taking the square root to obtain the root mean square value of the hot spot pixel gradient of the corresponding frame.

[0016] In a preferred embodiment, the volatility indicator combination is composed of the following data 1 and data 2:

[0017] Data 1 is calculated by averaging multiple temperature change rates within a preset fixed time window;

[0018] Data 1 is obtained by finding the maximum value of the local temperature gradient root mean square in multiple frames of images within a preset fixed time window;

[0019] To judge whether the temperature rise trend is in a stable state means:

[0020] Compare the volatility index combination with the preset stability reference index combination numerical range. If the volatility index combination falls within the preset stability reference index combination numerical range, the temperature rise trend is in a stable state; otherwise, the temperature rise trend is not in a stable state.

[0021] In a preferred embodiment, the lightweight deep network model deployed in the edge prediction module is constructed based on a temporal convolutional network or a long short-term memory network. The input data comes from the temperature evolution data volume output by the image conversion module. When calling the lightweight deep network model, the edge prediction module uses the temperature value sequence of the target area in multiple consecutive time steps as input data, where the temperature value sequence includes the original thermal imaging temperature data and first-order difference temperature change data of several frames before the current time point.

[0022] After receiving the input data, the lightweight deep network model extracts preset features of the time series structure and recursively generates a sequence of temperature prediction values ​​for the target area in multiple future time steps.

[0023] In a preferred embodiment, the abnormal activation index is generated by:

[0024] Perform a one-dimensional or two-dimensional Sobel operator on the temperature distribution of the target area in the current infrared thermal imaging image, calculate the temperature change value of each pixel in the gradient direction, and extract all local maximum gradient change values ​​within the target area based on a fixed-size sliding window. Select the global maximum value as the absolute value of the maximum hot spot gradient change in the current image frame;

[0025] Extract the corresponding maximum hot spot gradient absolute value from the target area in the previous m frames of the image, and perform arithmetic average operation to obtain the historical average change as a reference measurement;

[0026] The deviation is obtained by subtracting the historical average change from the absolute value of the maximum hot spot gradient change in the current image frame. The deviation is then normalized using the sum of the historical average change and a preset stability constant as the denominator. The stability constant is a non-zero positive number used to avoid numerical instability caused by the denominator approaching zero. Finally, the normalized result is compared with zero. If the result is less than zero, it is taken as zero. If the result is greater than zero, its original value is retained to determine the abnormal activation index.

[0027] In a preferred embodiment, the prediction deviation index is generated by:

[0028] Obtain n lightweight prediction curves generated by the lightweight deep network model during the first n calls, and simultaneously obtain the actual temperature rise trend data corresponding to each call. For each prediction curve and its corresponding actual temperature rise trend data, calculate the square of the difference between the predicted value and the actual value at multiple time steps, and obtain n residual square sums to form a residual sequence;

[0029] The residual sequence is divided into sliding windows of equal width. The maximum squared residual value is extracted in each window, and its frequency of occurrence in all windows is counted. The ratio of this frequency value to the total number of windows is defined as the local surge ratio. The maximum squared residual value of each window is multiplied by its corresponding local surge ratio, the sum is divided by the total number of windows to obtain the residual confidence weight.

[0030] The standard deviation of n residual square values ​​is calculated, and the difference between the maximum and minimum values ​​is calculated. The sum of the standard deviation and the range is used as the prediction stability factor. The residual confidence weight is multiplied by the prediction stability factor to obtain the prediction deviation index.

[0031] In a preferred embodiment, the trained machine learning model used in the prediction decision module is a fuzzy logic controller, which includes an input membership function group, a rule base, and an output inference mechanism. The input membership function group is used to receive the abnormal activation index and the prediction deviation index as input parameters, and convert the two input parameters into corresponding membership values ​​based on the set fuzzy linguistic variables.

[0032] The rule base presets multiple sets of logical reasoning rules, which are used to map the membership combinations to the output variable space, where the output variables include two types of judgment results: central model priority and edge model priority. The output reasoning mechanism calculates the activation intensity of each output variable based on the fuzzy reasoning results, and converts the output variable activation intensity into a clear judgment signal through the defuzzification method. The prediction decision module chooses to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model for subsequent temperature rise trend prediction based on the judgment signal.

[0033] In a preferred embodiment, the high-order multimodal temperature rise prediction model includes a temperature time series data processing network, an image feature encoding network and a modal fusion inference network. The temperature time series data processing network is used to receive the temperature evolution data body of the target area in multiple historical time steps. The image feature encoding network is used to extract the regional distribution characteristics and edge temperature rise morphological information in the current infrared thermal imaging image. The modal fusion inference network fuses the output data of the temperature time series data processing network and the image feature encoding network, constructs a multimodal input tensor and inputs it into the deep neural network model for joint modeling and future prediction, and outputs the temperature change trajectory of the target area in multiple future time steps.

[0034] Technical effects and advantages of the present invention:

[0035] This method uses an image conversion module to convert the hot zone temperature values ​​in continuous frame images captured by infrared thermal imaging equipment into a temperature evolution data volume with spatial positioning and time series properties. This constructs a structured spatiotemporal sequence foundation and provides high-quality, uniformly formatted data input for subsequent calculation modules. This approach not only enhances the ability to model the temporal continuity of temperature changes but also improves the targeted nature of spatial analysis, enabling the system to accurately extract the temperature rise behavior of target areas from the image level, effectively supporting subsequent feature extraction and trend analysis.

[0036] This invention uses a fluctuation feature extraction module to perform feature calculations on the temperature evolution data volume, forming a combination of volatility indicators that dynamically determine whether the temperature rise trend is stable. This allows for precise control of the timing of lightweight deep network model invocations in the edge prediction module. This design avoids the use of edge model predictions with limited computing resources when the temperature rise fluctuates dramatically, improving the reliability of the model prediction results and the overall operational efficiency of the system. It also reduces the computational load on edge devices under complex conditions.

[0037] The present invention features a prediction decision module that uses the anomaly activation index and prediction offset index generated by the anomaly characterization module as a basis for dynamic selection, either invoking a high-order model deployed in a remote center or continuing to use an edge model to predict future temperature change trajectories. This establishes an adaptive switching mechanism for prediction strategies. This mechanism enables on-demand allocation and decision execution of prediction paths, enabling the system to maintain high prediction accuracy and response efficiency under various thermal field change conditions, significantly improving the practicality and adaptability of the temperature rise trend warning system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0039] Figure 1 This is a schematic diagram of the thermal imaging temperature rise trend warning system based on spatiotemporal sequence prediction in the present invention. DETAILED DESCRIPTION

[0040] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0041] Reference Figure 1 The following examples were obtained:

[0042] Example 1:

[0043] Thermal imaging temperature rise trend warning system based on spatiotemporal sequence prediction, including image conversion module, fluctuation feature extraction module, edge prediction module, anomaly characterization module and prediction decision module;

[0044] The image conversion module is used to complete the conversion from infrared thermal imaging raw image data to structured prediction input data. Its specific functions are as follows: first, continuous frame images of the target area are obtained through the infrared thermal imaging device, thereby capturing the thermal distribution status of the device, environment, or area in real time; then, based on the target area in the image, the spatial coordinate information and the corresponding shooting timestamp in each frame are extracted, and a logical mapping relationship between space and time is established through this coordinate-time association; then, the temperature values ​​of the hot zones in the target area are further extracted from the image, organized into a temperature data sequence in chronological order, and combined with the spatial coordinates to generate a data set with spatial location identification and time evolution characteristics. Ultimately, the temperature evolution data body output by this module constitutes a data structure that has both spatial positioning attributes and time series characteristics. This is the spatiotemporal sequence structure required for subsequent processing and becomes the data foundation for the system's core prediction logic.

[0045] The fluctuation feature extraction module is responsible for extracting key feature quantities from the spatiotemporal sequence structure to determine whether the current temperature rise trend is stable. The function of this module is to perform numerical modeling and statistical analysis on the temperature evolution data body. First, the temperature change rate per unit time is calculated by changing the mean temperature of adjacent time points, thereby obtaining the temperature dynamic change rate that measures the speed of thermal dynamic change. Secondly, through image analysis, a pixel window is constructed with the target area as the center in each frame of the thermal image, and the gradient values ​​of all hot spot pixels are extracted, and their root mean square values ​​are calculated to obtain the local temperature gradient intensity that represents the stability of the hot spot structure. These two numerical indicators together constitute a volatility index combination, which is used to determine the continuity and intensity of the thermal change process. When this index combination is compared with the stability reference interval preset in the system, it can be concluded whether the current temperature rise state of the target area is in a stage of severe fluctuation or a relatively stable operation.

[0046] The edge prediction module is the front-end prediction path in the system. Its purpose is to quickly and efficiently predict short-term temperature rise trends when the thermal field is relatively stable. This module implements local prediction inference by invoking a lightweight deep network model deployed at the thermal imaging acquisition end. This module offers advantages such as high real-time performance, fast response, and low resource consumption. The model takes the temperature evolution data volume output by the image conversion module as input, extracts a series of temperature values ​​for the target area at multiple consecutive time points, and combines the raw temperature data from several frames before the current time point with their first-order difference data to form a unified input tensor. After receiving this input, the model extracts temporal structure features and predicts temperature trajectories for multiple future time steps using a time-recursive approach. The prediction outputs of the previous n models are saved as lightweight prediction curves for subsequent anomaly detection and stability assessment. The edge prediction module operates only when the current state is determined to be stable by a combination of volatility indicators. Therefore, it is activated only when high-complexity computations are not required.

[0047] The anomaly characterization module is a key module in the system for identifying whether there is a sudden surge in the current heat distribution and whether the prediction model is running stably. This module mainly generates two core indices for subsequent judgment: the anomaly activation index and the prediction offset index. Among them, the anomaly activation index obtains the degree of change of the hot spot in the current image frame through image gradient calculation, and compares it with the average change trend in the historical multi-frame image. Finally, it is mapped to a non-negative value through a normalization function to quantify whether the current degree of hot spot activation is higher than normal; the prediction offset index performs residual analysis on multiple lightweight prediction curves and their corresponding actual temperature rise trend data, and combines the window frequency and deviation amplitude to calculate the change trend of the model prediction credibility. The core role of this module is to promptly identify potential risks when hot spots surge or the prediction deviates seriously, and provide sufficient quantitative basis for the decision-making module.

[0048] The prediction decision module is responsible for dynamically determining whether to continue using the edge prediction path or switch to the more accurate central prediction path during system operation, based on the current thermal field state and prediction stability. This module uses the anomaly activation index and prediction offset index output by the anomaly characterization module as input variables and feeds them into a trained machine learning model. In the current technical solution, this machine learning model is a fuzzy logic controller. Through input membership transformation, rule base matching, and inference output, it maps the two index values ​​into discrete judgment results, typically including "continue edge prediction" or "switch to the central model." The judgment result output by this module directly determines whether to call the remote center's high-order multimodal temperature rise prediction model. Through this mechanism, the system achieves adaptive control of prediction strategies under different thermal field fluctuations, improving overall accuracy and resource utilization efficiency.

[0049] The image conversion module is responsible for standardizing and structuring infrared thermal imaging image data. It is the first module in the early warning system to receive raw image information and convert it into a data structure that can be directly used by subsequent models. The image conversion module internally includes an image frame preprocessing structure consisting of an image noise filtering component and a hotspot recognition component.

[0050] Among them, the function of the image noise filtering component is to suppress the image noise of the continuous frame images obtained by the infrared thermal imaging device. The infrared thermal imaging device is a device that obtains the infrared radiation intensity distribution of the target surface in the scene and can output a sequence of thermal imaging image frames. Due to interference from the device itself or the imaging environment, the original thermal map often contains thermal noise such as background thermal disturbances, photosensitivity errors, and edge interference, so the image data needs to be noise filtered. The image noise filtering component can be performed by edge-preserving median filtering, bilateral filtering, or thermal map intensity variance threshold filtering. The purpose is to remove high-frequency noise components without destroying the boundary characteristics of the hot spot structure, thereby improving image stability and subsequent recognition accuracy.

[0051] The heat zone recognition component is used to automatically extract the position boundary and center reference point of the target area in the image after noise suppression. The target area refers to a highly thermally sensitive area that is predefined or dynamically detected in the image, usually representing the equipment or area that needs to be monitored. The position boundary is the outer contour of the target determined by temperature threshold, heat density clustering, or image segmentation algorithm; the center reference point is the center pixel position of the area in the image coordinate system and can be used for spatial positioning and tracking. Based on the extraction results of the position boundary and reference point, the heat zone recognition component generates the spatial coordinate data of the target area in each frame of the image.

[0052] The spatial coordinate data represents the two-dimensional coordinate position of the target area within the image frame, including its boundary and center position parameters. This spatial coordinate data is mapped one-to-one with the image frame's capture timestamp, which records the specific time each frame was captured by the thermal imaging device. This spatial coordinate data and timestamp constitute a sequence of image frames with positioning information, serving as the standardized input data for the image conversion module before extracting temperature information.

[0053] After the image frame sequence is processed by the image conversion module, temperature values ​​are extracted for the hot zones of the target area within each frame. These hot zone temperatures are the equivalent temperature values ​​obtained by converting the infrared radiation intensity of each pixel within the region. This extraction is accomplished using a grayscale-to-temperature mapping model. The image conversion module then binds the extracted hot zone temperature values ​​from each frame to spatial coordinate data and timestamp parameters to form a structured time series temperature data set. This data set is a temperature evolution data volume with both spatial location and time series properties. The temperature evolution data volume is a data structure that records the temperature changes of the target area at multiple consecutive time points and is structured and stored in the form of a table, tensor, or time series array. The image conversion module ultimately outputs this temperature evolution data volume, which serves as the input for the subsequent fluctuation feature extraction module, which calculates the temperature dynamic change rate and local temperature gradient intensity, and further constructs a combination of volatility indicators.

[0054] The Fluctuation Feature Extraction module performs structured numerical analysis on the temperature evolution data output by the Image Conversion Module to quantify the dynamic characteristics of thermal changes in the target area. The module's computational objective is to extract two numerical quantities, one measuring the target area's temperature change rate over time and the other measuring the stability of the thermal gradient over space, thereby forming a combination of fluctuation indicators for state assessment.

[0055] The temperature evolution data volume is a collection of temperature values ​​recorded at multiple consecutive time points in the target area's hotspots. This data volume contains the association between the target area's temperature value, the corresponding timestamp, and the spatial coordinates in each frame. The fluctuation feature extraction module performs numerical calculations based on this data volume.

[0056] The temperature dynamic change rate reflects the overall temperature trend of the target area within a unit time. The temperature dynamic change rate is calculated by selecting temperature data from two adjacent time points in the temperature evolution data volume, calculating the mean of all temperature pixel values ​​within the target area at each time point, and then dividing the difference between the two temperature means by the time interval between the two time points. The resulting ratio is the temperature change rate within the current unit time. This value reflects whether the target area is showing a trend of continued warming, stabilization, or decline at the current stage, and directly quantifies the direction and magnitude of temperature change.

[0057] The local temperature gradient intensity is used to reflect the spatial gradient distribution of the hot spot structure of the target area in a single-frame image. The specific calculation method is: in each frame of the thermal image, a two-dimensional pixel window of a preset size is established with the central reference point of the target area as the center. The window limits the calculation range to ensure that only the inside of the target area is analyzed. Inside the window, the grayscale values ​​of all hot spot pixels are extracted, and gradient calculations are performed on these pixels (such as based on the Sobel operator or gradient difference method) to obtain the grayscale change values ​​of each pixel in the horizontal and vertical directions; then, all pixel gradient values ​​are squared, and the average of the square values ​​is calculated. Finally, the square root operation is performed on the average value to obtain the root mean square value of the hot spot pixel gradient corresponding to the frame image. This value represents the local gradient intensity of the hot spot area in the frame image, and is an important indicator for measuring the clarity and concentration of the hot spot edge.

[0058] The Sobel operator is a gradient operator commonly used in image processing to extract edge information. It is implemented by applying a local convolution kernel to a two-dimensional image pixel matrix. For a two-dimensional pixel window of the target area in a thermal image, convolution operations are applied in two directions to each pixel within it, extracting the intensity of grayscale changes in the horizontal (x-direction) and vertical (y-direction).

[0059] Specifically, the Sobel operator includes weighted convolution templates in two directions:

[0060] Horizontal template (Gx):

[0061] Vertical template (Gy):

[0062] Each target pixel and its surrounding 3×3 pixels form a sliding window, which is convolved with the two templates mentioned above to obtain the gradient values ​​Gx and Gy of the pixel in two directions. The square root of the sum of these two values ​​is taken to obtain the gradient amplitude G of the pixel: The G values ​​of all pixels are summed up, the average of their square values ​​is calculated and then the square root is taken to obtain the local temperature gradient intensity value of the target area in the current frame image (i.e., the root mean square value of the hot spot pixel gradient).

[0063] The gradient difference method is a numerical approximation method used to calculate the local variation trend of pixel grayscale values ​​in different directions in an image. In a two-dimensional image coordinate system, the grayscale gradient of each pixel in the target area is approximated in the row and column directions. For each pixel I(i,j), its gradient in the x-direction (row direction) and y-direction (column direction) can be calculated by the following difference method:

[0064] Approximate value of the x-direction gradient: Gx = I(i+1, j) - I(i-1, j);

[0065] Approximate value of the y-direction gradient: Gy = I(i, j+1) - I(i, j-1);

[0066] This method does not require a convolution kernel and is suitable for fast gradient estimation with low computational effort in edge computing environments. Get the gradient amplitude G of the pixel. For a complete two-dimensional pixel window, square the G values ​​of all pixels, take the average, and take the square root to get the root mean square value of the gradient of the hot spot area in the frame image, which is used as the local temperature gradient intensity of the frame.

[0067] The Sobel operator has high precision and clearer edge response, and is suitable for central analysis models that have high requirements for hot spot edge positioning. The gradient difference method has low computational complexity and is suitable for real-time processing. It can be deployed in the edge prediction module for rapid response.

[0068] These two values ​​constitute a volatility index combination, designated as Data 1 and Data 2. Data 1 is obtained by selecting a preset fixed time window (e.g., m consecutive image frames) and calculating the arithmetic mean of the temperature change rates calculated for each pair of adjacent time points within that window. This mean is Data 1. Data 2 is obtained by calculating the root mean square value of the local temperature gradient for each image frame within the same time window and selecting the maximum value as Data 2. This resulting volatility index combination represents the overall thermal behavior trend and local structural stability of the target area.

[0069] Determining whether the temperature rise trend is in a stable state means comparing the volatility index combination with the numerical range of the stability reference index combination pre-set in the system. The reference interval defines a double-threshold interval range, which is used to define under what circumstances the system can determine that the temperature rise state of the target area is stable. If the values ​​of data one and data two in the volatility index combination are both between the corresponding reference interval boundaries, it is determined that the current temperature rise trend is in a stable state; conversely, if any indicator exceeds the reference interval boundary, it is determined that the current temperature rise trend is not in a stable state. The output result of the fluctuation feature extraction module is a quantitative judgment signal of the current thermal change state, which will directly affect the activation decision of the subsequent edge prediction module or the abnormal characterization module.

[0070] The edge prediction module is a functional module in the system that performs local rapid prediction tasks when the thermal change trend in the target area is relatively stable. Its core component is a lightweight deep network model deployed on the thermal imaging acquisition end. This model is used to start when the volatility indicator combination is determined to be in a stable state, and is used to predict the temperature change trend of the target area in the next several time steps to meet the operation requirements of low latency and low computing overhead. The lightweight deep network model deployed in the edge prediction module refers to a time series modeling neural network model with a simplified structure, a low number of parameters, and adaptation to the operating conditions of edge computing devices. It can be constructed based on the temporal convolutional network (TCN) or long short-term memory network (LSTM) in the existing technology.

[0071] A temporal convolutional network (TCN) is a neural network architecture that uses one-dimensional convolutions to operate in a sliding manner across the time dimension. By stacking multiple convolutional layers, it can extract long-term and short-term dependencies in input sequences. Its core advantages lie in its ability to process input sequences in parallel, its fixed receptive field, and its ability to capture local-to-global temporal dependency patterns.

[0072] The Long Short-Term Memory Network (LSTM) is a variant of the Recurrent Neural Network (RNN) with internal gating mechanisms such as input gate, forget gate, and output gate. It can effectively learn long-term dependencies in time series and performs well in sequence prediction tasks with time delays or historical trends.

[0073] The lightweight deep network model is customized and constructed based on the basic structural characteristics of the temporal convolutional network or the long short-term memory network, using a lightweight compression strategy, a differential fusion input structure, and an edge computing adaptation method. The model parameters are controlled within the preset upper limit, and the temperature change prediction results of the target area in multiple future time steps are output through recursion.

[0074] Lightweight compression strategy: This strategy includes channel pruning, layer pruning, low-order weight quantization, weight sharing, and sparsification. Channel pruning evaluates the importance of output channels in convolutional layers or LSTM units, retaining the main channels and removing redundant connections. Layer pruning limits the model depth to 1-2 layers. Weights are converted to 8-bit fixed-point representation and run in the quantized inference engine of the edge chip. Sparse connections are also applied to the weight matrices between sequential units to compress the model size to a specified parameter limit (e.g., less than 500KB).

[0075] Differential Fusion Input Structure: The input data for the lightweight deep network model comes from the temperature evolution data volume output by the image conversion module. The temperature evolution data volume is a time-based data set that records the temperature changes of the target area's hot spots at multiple time points. It contains the hot spot temperature value extracted from each frame of the image, spatial positioning data, and corresponding timestamps.

[0076] When the edge prediction module calls the lightweight deep network model, it extracts a sequence of temperature values ​​of the target area over multiple consecutive time steps from the temperature evolution data volume as the model's input data. This input data consists of two parts: the original thermal imaging temperature data extracted from several frames before the current time point; and the first-order differential temperature change data consisting of the temperature differences between adjacent frames.

[0077] First-order differences represent the difference between the temperature at any point in time and the temperature at the previous point in time. They are a fundamental feature construction method in time series analysis, helping to improve the model's sensitivity to dynamic features such as fluctuation trends and temperature rise rates. The original and differenced series can be combined into a unified input tensor via channel-dimensional concatenation for subsequent modeling.

[0078] Edge Computing Adaptation: This model is deployed and run at the edge, adapting to the AI ​​chips, embedded GPUs, or NPU modules integrated into thermal imaging acquisition devices. The deployment process includes steps such as model structure conversion, quantized inference format conversion, and input tensor size normalization. The model is loaded through inference engines such as TFLite, ONNX, or TensorRT, and temperature series prediction is performed using a single sample stream to avoid high memory usage. During edge platform operation, the model is only called when the temperature rise trend is determined to be stable, further reducing energy consumption and load, and achieving low-power prediction closed-loop control at the edge of the thermal imaging system.

[0079] Recursive Prediction: After receiving the constructed input tensor, the model performs feature extraction on the time series structure. This includes conventional techniques such as convolutional feature extraction, gated activation, or time-weighted aggregation to detect trends, periodic fluctuations, or local anomalies. Without going into detail here, after feature extraction, the edge prediction module recursively outputs a sequence of predicted temperature values ​​for the target area over multiple future time steps.

[0080] The recursive method means that the model only outputs the predicted value of the next time step at a time, and uses the predicted result as part of the new round of input sequence, gradually updating the model input, and completing the generation of multi-step temperature trajectories without expanding the model volume. The final output result is the temperature rise prediction curve of the current target area in the future period of time. It can be used by the anomaly characterization module for prediction offset index calculation, and can also be directly used for front-end alarm visualization and early warning strategy linkage. In the edge prediction module, the lightweight deep network model uses a recursive method to output temperature predictions for multiple time steps in the future. The recursive method is a sequence generation strategy that feeds back the model's own prediction results and participates in the subsequent prediction process. It is used to solve the technical problem that the model structure itself only supports single-step output but the task goal is multi-step prediction. The processing flow includes the following steps: at the current time point t0, the last several frames of temperature data in the input sequence and the corresponding first-order difference data are input into the model, and the predicted temperature value for the first future time step t1 is output; the prediction result at time t1 is used as the new frame input, the input sequence is updated, and the next input tensor is formed; the model inference is re-performed on the updated tensor to generate the prediction result for the next time step t2; the above operation is repeated until all the predicted values ​​within the future target time range are obtained. The core advantage of this method is that it can adapt to multi-step prediction tasks of any length without changing the model structure. However, to ensure the prediction accuracy, the number of prediction steps is usually limited to a certain range, and the stability of the input structure can be maintained through a sliding window method.

[0081] The abnormal activation index is used to reflect whether the hot spot gradient of the target area in the current image frame has deviated significantly from the historical change trend. Its calculation is based on the hot spot change structure in the current image and historical images, and specifically includes the following steps:

[0082] Extracting the Current Hot Spot Gradient Change: A one-dimensional or two-dimensional Sobel operator is applied to the temperature distribution of the target area in the current infrared thermal image. The Sobel operator is an edge detection algorithm used in image processing. It uses convolution to extract the intensity of changes in grayscale or temperature values ​​in a specific direction (horizontally or vertically) within the image. If a one-dimensional Sobel operator is used, the image gradient is estimated in either the horizontal or vertical direction. If a two-dimensional Sobel operator is used, the gradient components in both directions are simultaneously acquired, and the gradient magnitude at each pixel is calculated using the square root of the sum of squares. The appropriate dimension can be selected based on the required computational accuracy. After performing the Sobel operation on all pixels in the target area, the temperature change value for each pixel in the gradient direction is obtained. A local search is then performed within the target area using a fixed-size sliding window. A sliding window is a subregion of a fixed width and height that slides across the image pixel matrix for local calculations. The gradient magnitudes of all pixels within the window are extracted, and the local maximum gradient change value within each window is found. After traversing the entire target area, the largest of all local maxima is taken, which is defined as the absolute value of the maximum hot spot gradient change in the current image frame. This value is used to measure the point where the hot spot edge changes most dramatically in the current frame and is a key indicator of hot spot activity.

[0083] Historical Average Change Extraction: Extract the maximum absolute value of the hot spot gradient change in each frame from the previous m frames of the target area, calculated using the same method as the current image. These values ​​form a numerical sequence of length m. The arithmetic mean is then performed on this numerical sequence, summing all historical maximum values ​​and dividing by the number of frames m to obtain an average value, defined as the historical average change. This historical average change reflects the baseline hot spot activity in the target area over a short period of time and serves as a comparison standard for determining whether the current frame is abnormal.

[0084] Normalized deviation calculation: Subtract the absolute value of the maximum hot spot gradient change calculated in the current image frame from the historical average change to obtain the difference between the two, which is defined as the deviation. The deviation reflects the deviation between the current intensity of the hot spot and the historical average state. In order to enhance the stability of the deviation value, a normalization operation is performed next. The sum of the historical average change and a preset stability constant is used as the denominator. The stability constant is a non-zero positive number with a value much smaller than the temperature gradient amplitude. The purpose is to prevent the mathematical error of dividing by zero when the historical average change is too small or even zero. The normalized result represents the standardized difference in the current hot spot activity relative to the historical baseline.

[0085] Non-negative processing and determination of the abnormal activation index: The normalized deviation result is compared with zero. If the normalized value is less than zero, it indicates that the current change is below the historical baseline and does not constitute an anomaly, so it is set to zero. If the value is greater than zero, it indicates that a significant surge has occurred in the current image frame, so this value is retained. The final result is defined as the abnormal activation index, which is a non-negative real number and is positively correlated with the degree of hot spot anomaly in the current image.

[0086] The prediction drift index is used to evaluate the stability and deviation of predictions generated by a lightweight deep network model over multiple consecutive calls. It is a dynamic monitoring indicator of the model's prediction performance. The calculation of this index relies on the prediction data generated by the edge prediction module at multiple time points and the corresponding real observation data. The overall calculation process includes the following three stages:

[0087] Obtain the n lightweight prediction curves generated by the lightweight deep network model during the first n calls. Each prediction curve represents a series of temperature changes in the target area over multiple future time steps, predicted based on the input data at a specific moment. Simultaneously, obtain the actual temperature rise trend data corresponding to these n calls—that is, the actual temperature change series of the target area within the same time range, collected after the prediction. For each prediction curve and its corresponding actual temperature rise trend data, pair them one-to-one by time step. Calculate the difference between the predicted and actual temperature values ​​at the same time point and square this difference to represent the squared residual value at that time point. Sum the squared residual values ​​across all time steps to obtain the squared residual corresponding to the prediction curve. Repeat this process until n squared residual sums are obtained, forming a residual sequence of length n, which reflects the cumulative prediction error of the edge model at multiple time points.

[0088] The residual sequence is partitioned into sliding windows of equal width. This means that the n residual sums of squares data are divided into several continuous subsequence windows of uniform length, each containing a fixed number of residual values. Within each sliding window, the maximum residual square value is extracted and used as the local maximum deviation for that window. The number of times each residual square value appears as the maximum value across all windows is counted, and its frequency of occurrence is calculated. This frequency is divided by the total number of windows to obtain the local surge ratio corresponding to each maximum residual square value. This ratio reflects the concentration of abnormal deviations within a certain forecast time period. The maximum residual square value within each sliding window is multiplied by its corresponding local surge ratio. All weighted residual values ​​are then summed and normalized by the total number of sliding windows. The result is defined as the residual confidence weight. This weight is used to comprehensively assess the severity and frequency of local high-deviation intervals in multiple model forecasts.

[0089] Statistical processing is performed on the n squared residual values ​​that constitute the residual sequence, and the standard deviation (indicating the degree of fluctuation of the residual value) and the range (i.e., the difference between the maximum and minimum values, reflecting the maximum dispersion) of the sequence are calculated respectively. The standard deviation and the range are summed up and defined as the prediction stability factor, which represents the degree of consistency of the model's prediction accuracy over different time periods. Finally, the residual confidence weight is multiplied by the prediction stability factor, and the resulting value is the prediction deviation index. This index is a non-negative real number. The larger the value, the worse the model prediction stability and the more severe the overall deviation. It is suitable for determining whether it is necessary to switch to the remote central model to perform high-precision prediction or trigger the model retraining mechanism.

[0090] For example, assume that the sum of squares of the residuals from ten predictions is [1.2, 0.9, 1.5, 3.8, 0.7, 2.2, 1.0, 0.8, 4.1, 1.3]. The data is divided into five windows, each containing two values. The maximum value in each window is [1.2, 1.5, 3.8, 2.2, 4.1]. The frequency of each value being the maximum value in the window is counted. Assume that the frequency is [1, 1, 1, 1, 1]. The surge ratio is 1 / 5 of the mean. The weighted total is (1.2 + 1.5 + 3.8 + 2.2 + 4.1) × 1 / 5 = 12.8 × 0.2 = 2.56. The standard deviation of the original residual series is set to 1.02, the range is 4.1 - 0.7 = 3.4, and the prediction stability factor is 1.02 + 3.4 = 4.42. The final prediction deviation index is: 2.56×4.42≈11.32, indicating that there is obvious inconsistency in the prediction results of the model across multiple calls.

[0091] The prediction and decision module determines whether to switch model paths during system operation based on the current target area's thermal dynamics and the predicted performance of the edge model. The trained machine learning model used in this module is a fuzzy logic controller. Fuzzy logic controllers are logical reasoning structures based on fuzzy set theory that map continuous-value inputs to linguistic rules. They are suitable for industrial decision-making scenarios involving uncertainty and fuzzy boundary judgments.

[0092] The fuzzy logic device includes an input membership function group, whose main function is to receive two core input parameters, namely: the anomaly activation index (reflecting the surge in hot spots in the target area); the prediction offset index (reflecting the prediction stability of the edge model). The input membership function group defines several membership functions for each input parameter, usually triangular functions, Gaussian functions or trapezoidal functions, which are used to map real-valued inputs to membership degrees. Each input parameter will be divided into several fuzzy language variable intervals, such as "low", "medium" and "high". When the value of the anomaly activation index or the prediction offset index falls into the corresponding interval, it will be attributed to the fuzzy language variable with a certain degree of membership. For example, an anomaly activation index of 0.45 may have both "medium (0.7)" and "high (0.3)" memberships.

[0093] The fuzzy logic controller has a predefined rule base containing several "if-then" logic rules based on fuzzy language. Each rule is defined as follows: If the anomaly activation index is [Level A] and the prediction deviation index is [Level B]; Then the output judgment is [Model Selection]. [Level A] and [Level B] correspond to the fuzzy language variables defined by the input membership function, and [Model Selection] selects one of two output variables: Edge model priority, which means the system continues to use the lightweight deep network model currently deployed at the thermal image acquisition end for prediction; Central model priority, which means the system switches to a remote center to call a high-order multimodal temperature rise prediction model to improve prediction accuracy. For example, a rule might be: If the anomaly activation index is "high" and the prediction deviation index is "high," then the central model is prioritized; or If the anomaly activation index is "low" and the prediction deviation index is "medium," then the edge model is prioritized. The rule base typically contains a complete set of rules covering all possible input combinations, such as a 3×3 rule set with nine rules.

[0094] After receiving the input parameters and calculating the membership, the fuzzy logic analyzer performs fuzzy inference on all matching rules based on the rule base. The fuzzy inference process calculates the activation strength of each rule's corresponding output variable, typically using a "min-implication" or "multiplication normalization" approach. For example, if the inputs are "medium (0.6)" and "high (0.5)," the corresponding rule activation strength is 0.6 × 0.5 = 0.3. All rule activation results that produce "edge model priority" and "center model priority" are combined to obtain the overall activation map for the two output variables.

[0095] After generating the fuzzy inference results, the system uses defuzzification to convert the activation strength into a deterministic numerical or categorical output. Common defuzzification methods include: the maximum membership method, which selects the output with the highest membership; and the centroid method, which calculates the centroid of the fuzzy output membership graph. In this system, the maximum membership method is used as the defuzzification mechanism to ensure rapid decision-making, directly selecting the output variable with the highest activation strength as the final decision signal.

[0096] The prediction decision module uses this defuzzification result, i.e., a clear judgment signal, to select a model call path. If the judgment result is "central model priority," the system automatically calls the high-order multimodal temperature rise prediction model deployed on the remote central platform. If the judgment result is "edge model priority," the system continues to use the lightweight deep network model deployed on the thermal imaging acquisition end to maintain the current prediction strategy. This mechanism constitutes the key decision-making logic that enables dynamic switching of prediction models and optimized response under real-time monitoring.

[0097] The high-order multimodal temperature rise prediction model includes a temperature time series data processing network, an image feature encoding network and a modal fusion inference network. The temperature time series data processing network is used to receive the temperature evolution data of the target area in multiple historical time steps. The image feature encoding network is used to extract the regional distribution characteristics and edge temperature rise morphological information in the current infrared thermal imaging image. The modal fusion inference network fuses the output data of the temperature time series data processing network and the image feature encoding network, constructs a multimodal input tensor and inputs it into the deep neural network model for joint modeling and future prediction, and outputs the temperature change trajectory of the target area in multiple future time steps.

[0098] The high-order multimodal temperature rise prediction model is the core prediction module deployed on the remote central computing platform in the system. It has multi-source data fusion processing capabilities and deep modeling capabilities, and is used to provide more accurate thermal imaging temperature rise trend prediction results when edge prediction strategies are insufficient. Specifically, it includes:

[0099] Temperature Time Series Data Processing Network: This network receives data on the temperature evolution of the target area over multiple historical time steps. This data is generated by the image conversion module and is a data structure with spatial positioning and time series properties. It contains the target area's hot zone temperature value, corresponding spatial coordinate data, and the capture timestamp for each frame of infrared thermal imaging.

[0100] The network is built based on a time series neural network, and its typical structure includes:

[0101] Multi-layer one-dimensional convolution stacking structure; or recursive network structure with multiple gated units; or Transformer encoder structure for self-attention to extract sequence features.

[0102] The network deeply encodes the temperature evolution trend in the time dimension, extracts long-term dependency features, periodic change features, and local temperature anomaly response patterns, and outputs an embedding vector representing the historical temperature rise behavior of the target area.

[0103] Image Feature Encoding Network: This network extracts regional distribution features and edge temperature rise information from the current infrared thermal image. The input is the latest infrared image acquired by the system, which contains the hot spot morphology and intensity distribution of the target area, as well as the difference between the hot spot and the surrounding background.

[0104] This network is a typical two-dimensional image coding neural network, which usually uses:

[0105] Convolutional neural network architectures (such as ResNet and MobileNet) incorporate edge-preserving modules and spatial attention mechanisms to enhance thermal boundary detection. Existing technologies are not detailed here. The output channels are feature maps or regional feature vectors containing spatial structural information. The network aims to extract the spatial distribution characteristics, edge contour mutation characteristics, and temperature gradient clustering characteristics of the target region in the current frame image, forming a structured representation in the static visual dimension.

[0106] Modal Fusion Inference Network: The modal fusion inference network is the core module of the high-order multimodal temperature rise prediction model. Its function is to effectively fuse the output data of the temperature time series data processing network and the image feature encoding network, build a unified input representation, and input it into the deep neural network for joint modeling and future prediction. Fusion methods include but are not limited to:

[0107] Vector concatenation, additive fusion, and a multiplicative attention mechanism are used; a cross-attention mechanism is used to reweight the features of the two modalities. Existing techniques are not described here. The final output is a unified multimodal input tensor with a dimensional structure suitable for use in subsequent predictive modeling modules. This fused multimodal input tensor is then fed into a deep prediction model, which can be a multilayer feedforward neural network, a time series modeling network with residual connections, or an encoder-decoder network. By jointly modeling time series trends and image spatial patterns, the model predicts temperature changes in the target area over multiple future time steps.

[0108] The final output is a temperature prediction sequence, representing the temperature evolution trend of the target area over several future time steps. The output structure is a one-dimensional time series or a two-dimensional temperature field tensor, which can be used as input for subsequent decision-making mechanisms to determine risk levels, compare warning thresholds, or identify thermal runaway trends.

[0109] For example, a temperature evolution data volume consisting of the last 10 heatmap frames of the target area is used as a time series input, while the current heatmap is also taken as the image input. These are fed into separate processing networks, resulting in a 64-dimensional time vector and a 128-dimensional spatial feature vector. The modal fusion inference network uses attention-weighted concatenation to construct a fusion tensor, which is then fed into a multi-step regression prediction network with a residual structure. The resulting output is the predicted temperature values ​​for the next five time points: [57.3, 58.9, 59.2, 60.1, 61.0] in °C. This result is used to determine whether the warning threshold set in subsequent operations (such as 60°C) is exceeded. If so, the system can trigger an abnormal status report.

[0110] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0111] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0112] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction is characterized by: It includes image conversion module, fluctuation feature extraction module, edge prediction module, abnormal characterization module and prediction decision module; The image conversion module acquires continuous frame images through infrared thermal imaging equipment, extracts the temperature value of the hot zone in each frame based on the spatial coordinates of the target area in the image and the shooting timestamp, constructs a temperature evolution data volume with spatial positioning and time series attributes, and forms a spatiotemporal sequence structure; The fluctuation feature extraction module calculates the temperature change rate per unit time and the root mean square of the hot spot pixel gradient in the thermal image based on the spatiotemporal sequence structure, respectively, to obtain the temperature dynamic change rate and local temperature gradient intensity, forming a combination of fluctuation indicators to determine whether the temperature rise trend is in a stable state; When the volatility indicator combination is in a stable range, the edge prediction module calls the lightweight deep network model deployed on the thermal imaging acquisition end to predict the temperature changes of the target area in multiple time steps in the future and use it as a basis for early warning. The prediction curves output by the first n calls of the lightweight deep network model constitute the lightweight prediction curve; The anomaly characterization module generates an anomaly activation index based on the difference between the absolute value of the maximum hot spot gradient change and the historical average change in the target area when the volatility indicator combination exceeds the stable range. It also generates a prediction offset index based on the mean square residual between the lightweight prediction curve and the actual temperature rise trend data corresponding to the previous n lightweight deep network model calls to evaluate the prediction stability of the edge model. The prediction decision module receives the abnormal activation index and the prediction offset index as input, and inputs them into the trained machine learning model. Based on the judgment results of the machine learning model, it chooses to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model, and outputs the temperature change trajectory of the target area in multiple time steps in the future as an early warning basis.

2. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 1 is characterized in that: The image conversion module has an image frame preprocessing structure consisting of an image noise filtering component and a hotspot recognition component. The image noise filtering component is used to suppress noise in continuous frame images acquired by the infrared thermal imaging device. The hotspot recognition component is used to automatically extract the position boundary and center reference point of the target area in the image and generate spatial coordinate data. The spatial coordinate data is mapped one-to-one with the shooting timestamp to form an image frame sequence with positioning information. After the image frame sequence is processed by the image conversion module, the temperature value of the hot zone in each frame is extracted and combined with the spatial coordinate data and the shooting timestamp parameters to generate a temperature evolution data volume with spatial positioning and time series attributes. The temperature evolution data volume stores the temperature change information of the target area at multiple consecutive time points in a structured manner. The image conversion module finally outputs the temperature evolution data volume as the input basis for the subsequent fluctuation feature extraction module to calculate the volatility indicator combination.

3. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 2 is characterized in that: The fluctuation feature extraction module performs numerical calculations based on the temperature evolution data in the spatiotemporal sequence structure. The temperature dynamic change rate is obtained by dividing the difference between the mean temperature of the target area at two adjacent time points in the temperature evolution data by the time interval value, forming a numerical expression reflecting the temperature change rate per unit time. The local temperature gradient intensity is obtained by establishing a two-dimensional pixel window with the target area as the center in each frame of the thermal image, extracting the grayscale value gradient of all hot spot pixels in the two-dimensional pixel window, and calculating the average of the sum of the squares of all pixel gradient values ​​and then taking the square root to obtain the root mean square value of the hot spot pixel gradient of the corresponding frame.

4. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 3 is characterized in that: The volatility indicator combination consists of the following data 1 and data 2: Data 1 is calculated by averaging multiple temperature change rates within a preset fixed time window; Data 1 is obtained by finding the maximum value of the local temperature gradient root mean square in multiple frames of images within a preset fixed time window; To judge whether the temperature rise trend is in a stable state means: Compare the volatility index combination with the preset stability reference index combination numerical range. If the volatility index combination falls within the preset stability reference index combination numerical range, the temperature rise trend is in a stable state; otherwise, the temperature rise trend is not in a stable state.

5. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 4 is characterized in that: The lightweight deep network model deployed in the edge prediction module is built based on a temporal convolutional network or a long short-term memory network. The input data comes from the temperature evolution data output by the image conversion module. When calling the lightweight deep network model, the edge prediction module uses the temperature value sequence of the target area in multiple consecutive time steps as input data, where the temperature value sequence includes the original thermal imaging temperature data and first-order difference temperature change data of several frames before the current time point. After receiving the input data, the lightweight deep network model extracts preset features of the time series structure and recursively generates a sequence of temperature prediction values ​​for the target area in multiple future time steps.

6. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 5 is characterized in that: The abnormal activation index is generated in the following way: Perform a one-dimensional or two-dimensional Sobel operator on the temperature distribution of the target area in the current infrared thermal imaging image, calculate the temperature change value of each pixel in the gradient direction, and extract all local maximum gradient change values ​​within the target area based on a fixed-size sliding window. Select the global maximum value as the absolute value of the maximum hot spot gradient change in the current image frame; Extract the corresponding maximum hot spot gradient absolute value from the target area in the previous m frames of the image, and perform arithmetic average operation to obtain the historical average change as a reference measurement; The deviation is obtained by subtracting the historical average change from the absolute value of the maximum hot spot gradient change in the current image frame. The deviation is then normalized using the sum of the historical average change and a preset stability constant as the denominator. The stability constant is a non-zero positive number used to avoid numerical instability caused by the denominator approaching zero. Finally, the normalized result is compared with zero. If the result is less than zero, it is taken as zero. If the result is greater than zero, its original value is retained to determine the abnormal activation index.

7. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 6 is characterized in that: The forecast deviation index is generated as follows: Obtain n lightweight prediction curves generated by the lightweight deep network model during the first n calls, and simultaneously obtain the actual temperature rise trend data corresponding to each call. For each prediction curve and its corresponding actual temperature rise trend data, calculate the square of the difference between the predicted value and the actual value at multiple time steps, and obtain n residual square sums to form a residual sequence; The residual sequence is divided into sliding windows of equal width. The maximum squared residual value is extracted in each window, and its frequency of occurrence in all windows is counted. The ratio of this frequency value to the total number of windows is defined as the local surge ratio. The maximum squared residual value of each window is multiplied by its corresponding local surge ratio, the sum is divided by the total number of windows to obtain the residual confidence weight. The standard deviation of n residual square values ​​is calculated, and the difference between the maximum and minimum values ​​is calculated. The sum of the standard deviation and the range is used as the prediction stability factor. The residual confidence weight is multiplied by the prediction stability factor to obtain the prediction deviation index.

8. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 7 is characterized in that: The trained machine learning model used in the prediction and decision module is a fuzzy logic controller, which includes an input membership function group, a rule base, and an output inference mechanism. The input membership function group is used to receive the abnormal activation index and the prediction deviation index as input parameters, and convert the two input parameters into corresponding membership values ​​based on the set fuzzy linguistic variables. The rule base presets multiple sets of logical reasoning rules, which are used to map the membership degree combination to the output variable space, where the output variables include two types of judgment results: central model priority and marginal model priority; The output inference mechanism calculates the activation intensity of each output variable based on the fuzzy inference results, and converts the output variable activation intensity into a clear judgment signal through the defuzzification method. The prediction decision module chooses to call the high-order multimodal temperature rise prediction model deployed in the remote center or continue to use the lightweight deep network model for subsequent temperature rise trend prediction based on the judgment signal.

9. The thermal imaging temperature rise trend warning system based on spatiotemporal series prediction according to claim 8 is characterized in that: The high-order multimodal temperature rise prediction model includes a temperature time series data processing network, an image feature encoding network and a modal fusion inference network. The temperature time series data processing network is used to receive the temperature evolution data of the target area in multiple historical time steps. The image feature encoding network is used to extract the regional distribution characteristics and edge temperature rise morphological information in the current infrared thermal imaging image. The modal fusion inference network fuses the output data of the temperature time series data processing network and the image feature encoding network, constructs a multimodal input tensor and inputs it into the deep neural network model for joint modeling and future prediction, and outputs the temperature change trajectory of the target area in multiple future time steps.

Images