Temperature threshold prediction method based on improved Transform

Abstract

The invention discloses a temperature threshold prediction method based on an improved Transform, which is used for solving the problems of tedious operation and frequent false alarm and missing alarm caused by manual setting of a threshold in a traditional alarm system. According to the invention, environment data are collected through a DTS temperature sensor, and data preprocessing is carried out in combination with sine-cosine coding. A deep learning model based on improved Transform is adopted as a core, the deep learning model comprises an input layer, an encoder module, a decoder module and an output layer, time sequence features in historical data are extracted through a sparse self-attention mechanism, the optimal threshold value of an alarm system is dynamically predicted, finally, dependence on manual setting is reduced, and the reliability of temperature monitoring is improved.

Description

Technical Field

[0001] The invention patent relates to a temperature threshold prediction algorithm, and in particular to a temperature threshold prediction method based on an improved Transformer. Background Art

[0002] With the rapid development of science and technology and society, temperature monitoring plays a vital role in many fields in many industrial production, equipment operation and other scenarios. Due to the different environments of different devices, affected by climate, indoor and outdoor factors, temperature monitoring is full of extremely high complexity. Traditional temperature monitoring systems mainly rely on manual settings or statistical principles to obtain thresholds, but they cannot cope with complex environmental changes, and often have excessive alarms or missed reports. The existing popular methods are mainly based on LSTM prediction methods, but the gating structure of LSTM increases the number of model parameters, resulting in more memory and computing power required during training and inference. When processing long sequences, it is limited by the recursive structure and computing efficiency, and the training speed is slow.

[0003] Therefore, there is an urgent need to develop a new temperature threshold prediction method to solve the above problems existing in the existing prediction methods. Summary of the invention

[0004] The purpose of the present invention is to solve the technical problems existing in the above-mentioned background technology. The present invention patent proposes a temperature threshold prediction method based on an improved Transformer. That is, the model is learned using the existing temperature data, and the temperature alarm threshold is predicted by the model. The model can be adapted according to different environmental conditions, and flexibly and intelligently predicts the appropriate alarm threshold, thereby greatly reducing the dependence on manual intervention and significantly reducing the probability of alarm errors, providing a reliable solution for temperature monitoring and management in related fields.

[0005] The technical solution of the present invention is: A temperature threshold prediction method based on an improved Transformer, characterized in that: the temperature threshold prediction method based on the improved Transformer comprises the following steps: Step 1: Data preparation and preprocessing, obtain historical data of DTS equipment operation over a period of time, and use historical data for analysis and modeling; Step 1.1: Use the DTS temperature sensor to collect temperature, location, sensor status data, time, battery power signal strength data, perform routine preprocessing on the data, clean the data to ensure that the data is clean and has no missing values, and generate historical data.

[0006] Step 1.2 processes historical data, divides the historical data into training set, validation set and test set, and adds noise for data enhancement.

[0007] The existing Transformer model is used as the initial model, the training set is used for Transformer model training, the validation set is used for Transformer model verification, and the test set is used for Transformer model testing, and finally the performance of the Transformer model is evaluated.

[0008] Step 2: Construct the input and output of the model, determine the representation of the input and output data and possible additional features; Step 2.1, use the temperature sequence in the historical data in step 1.1 as the input of the Transformer model, use the past N data to predict the future threshold, and the input data is a [N,1] temperature sequence; Step 2.2, the output can be a scalar (single threshold) indicating the temperature threshold that may require an alarm at some point in the future; Step 2.3: The actual value of the alarm threshold of the target output can be defined as a statistical indicator based on historical data, such as maximum value, minimum value, mean value, standard deviation, variance, range, skewness, and kurtosis.

[0009] Step 3: Use the Encoder module of Transformer to process the temperature time series data; Step 3.1: The input data is a time series of historical temperature data, and the temperature of each time step is mapped to a high-dimensional space through an embedding layer; Step 3.2: After mapping to high-dimensional space, add sine and cosine codes to represent the position information; Step 3.3: The Encoder captures the long-term dependencies in the historical temperature data through Locality-Sensitive Hashing Attention. Step 3.4: Using the multi-head attention mechanism, the model can calculate the relationship between different positions in parallel, thereby better understanding multiple aspects of the data.

[0010] Step 4: Train the model and determine the loss function and SGD, Adam, AdaDelta, and RMSprop optimizers used in training; Step 4.1 The alarm threshold is continuous, and the MSE loss function is used to train the model, that is, to minimize the difference between the predicted threshold and the true threshold; Step 4.2 uses the Adam optimizer, which can adaptively adjust the learning rate and is suitable for training deep learning models; Step 4.3 During training, ensure proper cross-validation to avoid overfitting; Step 4.4: Adjust the Transformer model’s hyperparameters, such as the number of layers, embedding dimension, feedforward network size, weight decay, hidden layer size, and learning rate, based on the evaluation metrics to adjust the model’s performance. Step 4.5 updates the model parameters through incremental learning or periodic retraining to ensure that the model adapts to the dynamic changes of different seasons and environmental conditions.

[0011] Step 5: Use the new real-time temperature data to input into the trained model to generate the alarm threshold and determine whether to trigger the alarm. Step 5.1 deploys the trained model to the alarm system to receive new data and predict thresholds in real time.

[0012] The advantages of the present invention are: 1. Accurate time series modeling capabilities The improved Transformer structure can better capture long-term dependencies and complex patterns in time series when processing temperature data. Since temperature data usually has seasonality and periodicity, the Transformer can effectively focus on relevant information in different time periods through the self-attention mechanism, thereby improving the accuracy of prediction.

[0013] 2. Enhanced nonlinear modeling capabilities Traditional time series models (such as ARIMA and LSTM) are sometimes limited in capturing nonlinear features, while the improved Transformer model can better capture the complex nonlinear relationships in the data through multi-layer self-attention mechanisms and fully connected layers. This enables it to handle more complex patterns in temperature threshold prediction, especially when environmental factors change in a complex manner.

[0014] 3. Effective integration of global information The Transformer's self-attention mechanism enables the model to effectively obtain information from each position of the input sequence without relying on local windows. This global information fusion capability helps the model consider the influencing factors of long time spans when predicting temperature thresholds, avoiding the limitations of relying only on local information.

[0015] 4. Flexible adaptive capabilities The improved Transformer method may include some flexible mechanisms (such as multi-scale feature extraction, improved loss function or adaptive optimization algorithm) to enable adaptive adjustment according to the actual characteristics of the data. For example, new regularization methods can be introduced to prevent overfitting, or the adaptability of the model can be enhanced by mixing multiple self-attention mechanisms.

[0016] 5. Efficient training and inference Compared with traditional recurrent neural networks (RNN) or long short-term memory networks (LSTM), the Transformer model usually has the advantage of parallel computing, making the training process more efficient. In addition, due to the parallelism of its structure, the reasoning process is relatively fast, which can meet the real-time requirements in temperature threshold prediction.

[0017] 6. Strong generalization ability The temperature threshold prediction method based on the improved Transformer can better adapt to a variety of different input features and learn different temperature patterns. This generalization ability enables the model to effectively predict temperature thresholds in a variety of different environments or scenarios, especially when facing changes in data distribution or seasonal changes, it can still maintain a high prediction accuracy.

[0018] 7. Good scalability The structure of this method has good scalability and can be expanded to more complex prediction tasks as the amount of data increases or the demand changes. By adjusting parameters such as the number of model layers and the number of attention heads, large-scale data sets or high-dimensional data can be better processed, further improving the prediction ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which constitute a part of the specification of the present invention, are used to provide a further understanding of the present invention.

[0020] Figure 1 It is a brief structural diagram of the model used in the patent of this invention.

[0021] Figure 2 It is a schematic diagram of the system prediction alarm threshold flow provided by the patent of this invention. DETAILED DESCRIPTION

[0022] This implementation provides a temperature threshold prediction method based on an improved Transformer. The specific implementation method is described below: Step 1: Data preparation and preprocessing Step 1.1 Use the DTS temperature sensor to collect temperature, location, sensor status data, time, battery power signal strength data, and generate complete time series data. The data record should include the temperature values, timestamps, and location information of multiple sensors to ensure data diversity and representativeness. Step 1.2: Perform routine preprocessing on the collected data, check and fill in missing values, and ensure that there are no missing data or abnormal values ​​collected. If there are missing values ​​in the data, interpolation can be used to fill them. Filter outliers generated by the collection.

[0023] Step 1.3: Divide the historical temperature data into three subsets: training set, validation set, and test set. The training set is used to train the model, the validation set is used to verify the model and tune hyperparameters, and the test set is used to evaluate the performance of the final model. Step 1.4 can significantly enhance the robustness and generalization ability of the model by adding noise or disturbance to the temperature data. The addition of noise simulates the uncertainty in the actual environment, while the disturbance creates diverse training data, helping the model to better cope with complex situations. This method is an effective means to improve the performance of deep learning models in real-world applications, especially when dealing with data with noisy or uncertain characteristics. This improves the robustness and generalization ability of the model.

[0024] Step 2: Construct the model input and output Step 2.1: Use the historical temperature sequence as the input of the Transformer model. Assume that the past N data are used to predict the future alarm threshold, and the input data is a temperature sequence of size [N, 1]; Step 2.2 The output of the model is an alarm threshold, which is a scalar that represents the alarm threshold at a certain point in the future; In step 2.3, the alarm threshold of the target output can be set according to actual needs, and the threshold can be set based on the statistical indicators of the temperature series (such as maximum value, minimum value, mean value, etc.).

[0025] Step 3: Use the Transformer encoder module to process temperature time series data Step 3.1: Map the temperature data to a high-dimensional space through an embedding layer to enhance the feature expression capability of the data; Step 3.2 After the embedding layer, add sine and cosine encoding to represent the position information of each data point in the time series. Since Transformer lacks traditional sequence modeling capabilities, position encoding helps the model understand the order of the data; Step 3.3 The Encoder module of the Transformer module processes the input time series data and captures long-term dependencies through the self-attention mechanism. LSH Attention is used to efficiently process longer time series and enhance the computational efficiency of the model when processing large-scale data. Using the multi-head attention mechanism, the model can calculate the relationship between different positions in parallel, thereby better understanding the multiple levels of time series data.

[0026] Step 4: Train the model Step 4.1 Since the alarm threshold is continuous, MSE can be used as the loss function. MSE aims to minimize the difference between the alarm threshold predicted by the model and the true threshold. Step 4.2 uses the Adam optimizer, which has the ability of adaptive learning rate, is suitable for training deep learning models, and can improve the convergence speed of the model; Step 4.3: Evaluate the performance of the model through cross-validation and other methods to avoid overfitting. During the training process, you can use the validation set to adjust the hyperparameters (such as learning rate, batch size, number of layers, etc.). Regularly save the intermediate state of the model to ensure the stability of the model and a good training process; Step 4.4 Through experiments, continuously adjust the hyperparameters of the Transformer model, including the number of layers, embedding dimension, feedforward network size, weight decay, number of attention heads, hidden layer size, and learning rate, to find the optimal model configuration; Step 4.5 Considering environmental changes, the model may need to be incrementally learned or retrained regularly. For example, at regular intervals or as the seasons change, the model's parameters can be updated to adapt to new temperature change patterns.

[0027] Step 5: Real-time application and alarm triggering Step 5.1 deploys the trained Transformer model to the actual alarm system to receive new data provided by the temperature sensor in real time; Step 5.2: Input the new temperature data into the trained model, and the model will predict the future alarm threshold based on the current historical data. Step 5.3 compares the alarm threshold output by the model with the real-time temperature data to determine whether to trigger an alarm. For example, if the real-time temperature exceeds the predicted alarm threshold, the system will automatically trigger the alarm mechanism.

[0028] The temperature threshold prediction method based on the improved Transformer is used to solve the problems of cumbersome operation and frequent false alarms and missed alarms caused by manually setting thresholds in traditional alarm systems. The patent of this invention collects environmental data (including temperature, time, location, etc.) through the DTS temperature sensor, and performs data preprocessing in combination with sine-cosine coding. The core adopts a deep learning model based on the improved Transformer, including an input layer, an encoder module, a decoder module, and an output layer. The sparse self-attention mechanism extracts time series features from historical data, and dynamically predicts the optimal threshold of the alarm system, ultimately reducing dependence on manual settings and improving the reliability of temperature monitoring.

Claims

1. A temperature threshold prediction method based on an improved Transformer, characterized in that: The temperature threshold prediction method based on the improved Transformer comprises the following steps: Step 1: Data preparation and preprocessing, obtain historical data of DTS equipment operation over a period of time, and use historical data for analysis and modeling; Step 2: Construct the input and output of the model, determine the representation of the input and output data and possible additional features; Step 3: Use the Encoder module of Transformer to process the temperature time series data; Step 4: Train the model and determine the loss function and SGD, Adam, AdaDelta, and RMSprop optimizers used in training; Step 5: Use the new real-time temperature data to input into the trained model to generate the alarm threshold and determine whether to trigger the alarm.

2. According to claim 1, a temperature threshold prediction method based on improved Transformer is characterized in that: The step 1 comprises the following steps: Step 1.1, collect temperature, location, sensor status data, time, battery power signal strength data through DTS temperature sensor, perform routine preprocessing on the data, clean the data to ensure that the data is clean and has no missing values, and obtain historical data; Step 1.2: Process historical data, divide the historical data into training set, validation set and test set, and add noise for data enhancement; The existing Transformer model is used as the initial model, the training set is used for Transformer model training, the validation set is used for Transformer model verification, and the test set is used for Transformer model testing, and finally the performance of the Transformer model is evaluated.

3. The temperature threshold prediction method based on improved Transformer according to claim 1 is characterized in that: The step 2 comprises the following steps: Step 2.1: Use the temperature sequence in the historical data in step 1.1 as the input of the Transformer model, and use the past N data to predict the future threshold. The input data is a [N, 1] temperature sequence. Step 2.2, the output can be a scalar, indicating the temperature threshold that may require an alarm at a certain point in the future; Step 2.3: The actual value of the alarm threshold of the target output can be defined as a statistical indicator based on historical data, such as maximum value, minimum value, mean value, standard deviation, variance, range, skewness, and kurtosis.

4. The temperature threshold prediction method based on improved Transformer according to claim 1 is characterized in that: The step 3 comprises the following steps: Step 3.1: The input data is a time series of historical temperature data, and the temperature of each time step is mapped to a high-dimensional space through an embedding layer; Step 3.2: After mapping to high-dimensional space, add sine and cosine codes to represent the position information; Step 3.3: The Encoder module of Transformer captures the long-term dependencies in the historical temperature data through Locality-Sensitive Hashing Attention. Step 3.4: Using the multi-head attention mechanism, the model can calculate the relationship between different positions in parallel, thereby better understanding multiple aspects of the data.

5. The temperature threshold prediction method based on improved Transformer according to claim 1 is characterized in that: The step 4 comprises the following steps: Step 4.1: The alarm threshold is continuous, and the MSE loss function is used to train the model, that is, to minimize the difference between the predicted threshold and the true threshold; Step 4.2: Use the Adam optimizer, which can adaptively adjust the learning rate and is suitable for training deep learning models. Step 4.3: During training, ensure proper cross-validation to avoid overfitting. Step 4.4: According to the evaluation indicators, adjust the hyperparameters of the transformer model, such as the number of layers, hidden layer size, learning rate, etc., to adjust the performance of the model; Step 4.5: Update the model parameters through incremental learning or periodic retraining to ensure that the model adapts to the dynamic changes of different seasons and environmental conditions.

6. The temperature threshold prediction method based on improved Transformer according to claim 1 is characterized in that: The step 5 comprises the following steps: Step 5.1: Deploy the trained model to the alarm system to receive new data and predict thresholds in real time.

Images