A temperature threshold prediction method based on improved transformer

Abstract

The patent discloses a temperature threshold prediction method based on improved Transform, which is used to solve the problem of frequent false alarms and missed alarms caused by manual threshold setting in traditional alarm systems. The patent collects environmental data through a DTS temperature sensor and preprocesses the data in combination with sine-cosine coding. The core uses a deep learning model based on improved Transform, including an input layer, an encoder module, a decoder module and an output layer. Through the sparse self-attention mechanism, the time series features in the historical data are extracted, the optimal threshold of the alarm system is dynamically predicted, and finally the dependence on manual setting is reduced and the reliability of temperature monitoring is improved.

Description

Technical Field

[0001] This invention relates to temperature threshold prediction algorithms, and in particular to a temperature threshold prediction method based on an improved Transformer. Background Technology

[0002] With the rapid development of science and technology and society, temperature monitoring plays a crucial role in many fields, including industrial production and equipment operation. Due to the varying environments in which different equipment operates and the influence of climate, indoor and outdoor factors, temperature monitoring is highly complex. Traditional temperature monitoring systems mainly rely on manual setting or statistical principles to obtain thresholds, but they cannot cope with complex environmental changes, often resulting in over-alarms or under-alarms. Current popular methods are primarily based on LSTM prediction methods, but the gating structure of LSTM increases the number of model parameters, requiring more memory and computing power during training and inference. When processing long sequences, the training speed is slow due to limitations in recursion structure and computational efficiency.

[0003] Therefore, there is an urgent need to develop a new temperature threshold prediction method to solve the above-mentioned problems of existing prediction methods. Summary of the Invention

[0004] The purpose of this invention is to address the technical problems existing in the background art by proposing a temperature threshold prediction method based on an improved Transformer. This method utilizes existing temperature data to learn from a model, which then predicts the temperature alarm threshold. This model can adapt to different environmental conditions, flexibly and intelligently predicting appropriate alarm thresholds, thereby greatly reducing reliance on manual intervention and significantly decreasing the probability of alarm errors, providing a reliable solution for temperature monitoring and management in related fields.

[0005] The technical solution of this invention is:

[0006] A temperature threshold prediction method based on an improved Transformer, characterized in that: the temperature threshold prediction method based on an improved Transformer includes the following steps:

[0007] Step 1: Data preparation and preprocessing. Obtain historical data of DTS equipment operation over a period of time, and use the historical data for analysis and modeling.

[0008] Step 1.1 Collect temperature, location, sensor status data, time, battery power signal strength data using a DTS temperature sensor. Perform routine preprocessing on the data and clean the data to ensure that the data is clean and has no missing values, thus obtaining historical data.

[0009] Step 1.2 Process historical data by dividing it into training, validation, and test sets, and adding noise for data augmentation.

[0010] The existing Transformer model is used as the initial model. The training set is used for training the Transformer model, the validation set is used for validating the Transformer model, and the test set is used for testing the Transformer model. Finally, the performance of the Transformer model is evaluated.

[0011] Step 2: Construct the input and output of the model, determine the representation of the input and output data and possible additional features; Step 2.1: Use the temperature sequence in the historical data in Step 1.1 as the input of the Transformer model, and use the past N data to predict the future threshold. The input data is a temperature sequence of [N,1].

[0012] Step 2.2: The output can be a scalar (single threshold), representing the temperature threshold that may trigger an alarm at some point in the future;

[0013] Step 2.3: The true value of the alarm threshold output by the target can be defined as a statistical indicator based on historical data, such as maximum value, minimum value, mean, standard deviation, variance, range, skewness, and kurtosis.

[0014] Step 3: Process the temperature time series data using the Transformer's Encoder module;

[0015] Step 3.1: The input data is a time series of historical temperature data. The temperature at each time step is mapped to a high-dimensional space through an embedding layer.

[0016] Step 3.2: After mapping to a high-dimensional space, add sine and cosine codes to represent the position information;

[0017] Step 3.3: The Encoder part uses Locality-Sensitive Hashing Attention to capture long-term dependencies in historical temperature data;

[0018] Step 3.4: By utilizing the multi-head attention mechanism, the model can compute the relationships between different positions in parallel, thereby gaining a better understanding of multiple aspects of the data.

[0019] Step 4: Train the model and determine the loss function and optimizers (SGD, Adam, AdaDelta, RMSprop) to be used for training.

[0020] Step 4.1 The alarm threshold is continuous. The MSE loss function is used to train the model, that is, to minimize the difference between the predicted threshold and the true threshold.

[0021] Step 4.2 uses the Adam optimizer, which can adaptively adjust the learning rate and is suitable for training deep learning models;

[0022] Step 4.3 During training, ensure appropriate cross-validation is performed to avoid overfitting;

[0023] Step 4.4 Based on the evaluation metrics, adjust the hyperparameters of the Transformer model, such as the number of layers, embedding dimension, feedforward network size, weight decay, hidden layer size, and learning rate, to adjust the model's performance.

[0024] Step 4.5 Update the model parameters through incremental learning or periodic retraining to ensure that the model adapts to dynamic changes in different seasons and environmental conditions.

[0025] Step 5: Input the new real-time temperature data into the trained model to generate alarm thresholds and determine whether to trigger an alarm.

[0026] Step 5.1 Deploy the trained model to the alarm system to receive new data in real time and predict thresholds.

[0027] The advantages of this invention are:

[0028] 1. Accurate time series modeling capability

[0029] The improved Transformer architecture can better capture long-term dependencies and complex patterns in time series data when processing temperature data. Since temperature data often exhibits seasonality and periodicity, the Transformer can effectively focus on relevant information across different time periods through its self-attention mechanism, thereby improving prediction accuracy.

[0030] 2. Enhanced nonlinear modeling capabilities

[0031] Traditional time series models (such as ARIMA and LSTM) are sometimes limited in capturing nonlinear features, while the improved Transformer model can better capture complex nonlinear relationships in data through multi-layer self-attention mechanisms and fully connected layers. This allows it to handle more complex patterns in temperature threshold prediction, especially when environmental factors are complex and changing.

[0032] 3. Effective integration of global information

[0033] The Transformer's self-attention mechanism enables the model to effectively extract information from every position in the input sequence without relying on local windows. This global information fusion capability helps the model consider factors over a long time span when predicting temperature thresholds, avoiding the limitations of relying solely on local information.

[0034] 4. Flexible adaptability

[0035] Improved Transformer methods may incorporate flexible mechanisms (such as multi-scale feature extraction, improved loss functions, or adaptive optimization algorithms) to adaptively adjust to the specific characteristics of the data. For example, overfitting can be prevented by introducing new regularization techniques, or the model's adaptability can be enhanced by combining multiple self-attention mechanisms.

[0036] 5. Efficient training and reasoning

[0037] Compared to traditional recurrent neural networks (RNNs) or long short-term memory networks (LSTMs), Transformer models typically offer the advantage of parallel computation, making the training process more efficient. Furthermore, due to the parallelizability of its structure, the inference process is also relatively fast, meeting the real-time requirements of temperature threshold prediction.

[0038] 6. Strong generalization ability

[0039] The temperature threshold prediction method based on the improved Transformer can adapt well to a variety of different input features and learn different temperature patterns. This generalization ability enables the model to effectively predict temperature thresholds in various environments or scenarios, and it can maintain high prediction accuracy, especially when faced with changes in data distribution or seasonality.

[0040] 7. Good scalability

[0041] This method boasts excellent scalability, allowing it to be extended to more complex prediction tasks as the amount of data increases or requirements change. By adjusting parameters such as the number of model layers and attention heads, it can better handle large-scale datasets or high-dimensional data, further enhancing prediction capabilities. Attached Figure Description

[0042] The accompanying drawings, which form part of this patent, are used to provide a further understanding of this patent.

[0043] Figure 1 This is a simplified structural diagram of the model used in this invention patent.

[0044] Figure 2This is a schematic diagram of the system's predictive alarm threshold process provided by this invention patent. Detailed Implementation

[0045] This implementation provides a temperature threshold prediction method based on an improved Transformer. The specific implementation method is described below:

[0046] Step 1: Data Preparation and Preprocessing

[0047] Step 1.1 Collect temperature, location, sensor status data, time, battery power signal strength data using DTS temperature sensors to generate complete time series data. The data record should include temperature values, timestamps, and sensor location information from multiple sensors to ensure data diversity and representativeness.

[0048] Step 1.2 Perform routine preprocessing on the collected data, checking and imputing missing values ​​to ensure there are no missing or outlier values. If missing values ​​are present, interpolation can be used to impute them. Filter outliers generated during the data collection process.

[0049] Step 1.3 divides the historical temperature data into three subsets: training set, validation set, and test set. The training set is used to train the model, the validation set is used for model validation and hyperparameter tuning, and the test set is used to evaluate the performance of the final model.

[0050] Step 1.4: By adding noise or perturbations to the temperature data, the robustness and generalization ability of the model can be significantly enhanced. Adding noise simulates the uncertainty in the real-world environment, while perturbations create diverse training data, helping the model better cope with complex situations. This method is an effective way to improve the performance of deep learning models in real-world applications, especially when dealing with data that is noisy or uncertain. This improves the model's robustness and generalization ability.

[0051] Step 2: Construct the model's inputs and outputs

[0052] Step 2.1 Use the historical temperature sequence as input to the Transformer model. Assume that N past data points are used to predict future alarm thresholds, and the input data is a temperature sequence of size [N,1].

[0053] Step 2.2 The output of the model is an alarm threshold, which is a scalar representing the alarm threshold at a future point in time.

[0054] Step 2.3 The alarm threshold for the target output can be set according to actual needs. You can choose to set the threshold based on statistical indicators of temperature sequence (such as maximum value, minimum value, average value, etc.).

[0055] Step 3: Process temperature time series data using the Transformer's encoder module.

[0056] Step 3.1 Maps the temperature data to a high-dimensional space through an embedding layer to enhance the data's feature representation capabilities;

[0057] Step 3.2: After the embedding layer, sine and cosine encoding is added to represent the positional information of each data point in the time series. Since the Transformer lacks traditional sequence modeling capabilities, positional encoding helps the model understand the order of the data;

[0058] Step 3.3: The Encoder module of the Transformer module processes the input time-series data, capturing long-term dependencies through a self-attention mechanism. LSH Attention is employed to efficiently process longer time series, enhancing the model's computational efficiency when handling large-scale data. Utilizing a multi-head attention mechanism, the model can compute relationships between different positions in parallel, thereby better understanding multiple levels of the time-series data.

[0059] Step 4: Train the model

[0060] Step 4.1 Since the alarm threshold is continuous, MSE can be used as the loss function. MSE aims to minimize the difference between the alarm threshold predicted by the model and the true threshold.

[0061] Step 4.2 uses the Adam optimizer, which has the ability to adapt the learning rate, making it suitable for training deep learning models and improving the convergence speed of the model;

[0062] Step 4.3: Evaluate the model's performance using methods such as cross-validation to avoid overfitting. During training, the validation set can be used to adjust hyperparameters (such as learning rate, batch size, number of layers, etc.). Regularly save the model's intermediate states to ensure model stability and a smooth training process.

[0063] Step 4.4 Through experiments, continuously adjust the hyperparameters of the Transformer model, including the number of layers, embedding dimension, feedforward network size, weight decay, number of attention heads, hidden layer size, and learning rate, to find the optimal model configuration;

[0064] Step 4.5 Considering environmental changes, the model may need to undergo incremental learning or periodic retraining. For example, the model's parameters can be updated at regular intervals or with seasonal changes to adapt to new temperature variation patterns.

[0065] Step 5: Real-time application and alarm triggering

[0066] Step 5.1 Deploy the trained Transformer model into the actual alarm system to receive new data from the temperature sensor in real time;

[0067] Step 5.2 Input the new temperature data into the trained model. The model will predict the future alarm threshold based on the current historical data.

[0068] Step 5.3 compares the alarm threshold output by the model with the real-time temperature data to determine whether to trigger an alarm. For example, if the real-time temperature exceeds the predicted alarm threshold, the system will automatically trigger the alarm mechanism.

[0069] This temperature threshold prediction method based on an improved Transformer addresses the problems of cumbersome operation and frequent false alarms / missed alarms caused by manual threshold setting in traditional alarm systems. This invention uses a DTS temperature sensor to collect environmental data (including temperature, time, location, etc.) and performs data preprocessing using sine-cosine coding. The core employs a deep learning model based on an improved Transformer, including an input layer, encoder module, decoder module, and output layer. It extracts time-series features from historical data through a sparse self-attention mechanism to dynamically predict the optimal threshold for the alarm system, ultimately reducing reliance on manual setting and improving the reliability of temperature monitoring.

Claims

1. An improved Transformer-based temperature threshold prediction method, characterized in that: The improved Transformer-based temperature threshold prediction method comprises the following steps: Step 1: data preparation and preprocessing, obtaining historical data of DTS device running in a period of time, analyzing and modeling using historical data; Step 2: input and output of model construction, determining the representation form of input and output data and possible additional features; Step 3: using the Encoder module of Transformer to process temperature time series data; Step 4: training the model, determining the loss function used for training and the SGD, Adam, AdaDelta, RMSprop optimizers; Step 5: inputting new real-time temperature data into the trained model to generate an alarm threshold and determine whether to trigger an alarm; The step 1 comprises the following steps: Step 1.1, collecting temperature, location, sensor state data, time, battery power signal strength data by DTS temperature sensor, performing conventional preprocessing on the data, cleaning the data to ensure that the data is clean and has no missing values, and preparing historical data; Step 1.2, processing historical data, dividing historical data into training set, validation set and test set, and adding noise for data enhancement; The existing Transformer model is used as the initial model, the training set is used for Transformer model training, the validation set is used for Transformer model validation, and the test set is used for Transformer model testing to finally evaluate the performance of the Transformer model; The step 2 comprises the following steps: Step 2.1, taking the temperature sequence in the historical data in step 1.1 as the input of the Transformer model, using the past N data to predict the threshold value in the future, and the input data is a temperature sequence of [N, 1]; Step 2.2, the output can be a scalar, representing the temperature threshold value that may need to be alarmed at a future time point; Step 2.3, the true value of the alarm threshold of the target output can be defined based on statistical indicators of historical data, including maximum value, minimum value, mean value, standard deviation, variance, range, skewness, kurtosis; The step 3 comprises the following steps: Step 3.1, the input data is the time series of historical temperature data, and the temperature at each time step is mapped to a high-dimensional space through an embedding layer; Step 3.2, after mapping to a high-dimensional space, add sine and cosine coding to represent position information; Step 3.3, the Encoder module of Transformer captures long-term dependencies in historical temperature data through Locality-Sensitive Hashing Attention; Step 3.4, using multi-head attention mechanism, the model can parallelly compute the relationship between different positions, so as to better understand multiple aspects of the data.

2. The improved Transformer-based temperature threshold prediction method according to claim 1, wherein: The step 4 comprises the following steps: Step 4.1, the alarm threshold is continuous, and the MSE loss function is used to train the model, that is, to minimize the difference between the predicted threshold and the true threshold; Step 4.2, use Adam optimizer, which can adaptively adjust the learning rate, suitable for training deep learning model; Step 4.3, during the training process, ensure proper cross-validation to avoid overfitting; Step 4.4, according to the evaluation index, adjust the hyperparameters of the transformer model, including the number of layers, the size of the hidden layer, and the learning rate, to adjust the performance of the model; Step 4.5, update the model parameters through incremental learning or periodic retraining to ensure that the model adapts to the dynamic changes of different seasons and environmental conditions.

3. The improved Transformer-based temperature threshold prediction method of claim 1, wherein: The step 5 includes the following steps: Step 5.1, deploy the trained model to the alarm system, receive new data in real time and predict the threshold value.

Images