Intelligent temperature control device and method based on LSTM-DQN cascaded network

Abstract

This application relates to the field of temperature control technology, specifically an intelligent temperature control device and method based on an LSTM-DQN cascaded network. By using the LSTM network to predict future temperature trends, the DQN network can respond in advance. Experimental results show that the control scheme of this invention can reduce the overshoot during the heating process from 15-20% in traditional PID control to less than 5%, effectively avoiding material damage caused by excessive temperature. Through the design of a multi-objective reward function and the introduction of LSTM prediction information, this device can control steady-state temperature fluctuations within ±0.5℃, significantly better than the ±3-5℃ of traditional PID control, meeting the requirements of high-precision applications such as lithium battery vacuum ovens. The continuous online learning capability of the DQN network enables the system to adapt to changes in operating conditions such as material batch variations, ambient temperature fluctuations, and heating element aging. Unlike traditional PID control, which is tuned once and used for life, the control strategy of this invention is continuously optimized with the increase of the number of runs, resulting in continuous improvement in control accuracy.

Description

Technical Field

[0001] This application relates to the field of temperature control technology, specifically an intelligent temperature control device and method based on an LSTM-DQN cascaded network. Background Technology

[0002] In industrial heating equipment such as lithium battery vacuum ovens, hot press molding machines, and plastic extruders, temperature control accuracy directly affects product quality and production efficiency. Currently, the temperature control of multi-zone heating equipment mainly adopts the following technical solutions:

[0003] (1) Traditional PID control. PID (proportional-integral-derivative) controllers are widely used due to their simple structure and ease of implementation. In PLCs, the PID algorithm is implemented in the form of function blocks, and the parameters Kp, Ti, and Td are manually tuned to adapt to the thermal characteristics of different temperature zones. Traditional PID control has drawbacks such as parameter tuning relying on manual experience and inability to adapt to changes in operating conditions.

[0004] (2) Self-tuning PID controller. Some high-end temperature controllers provide an auto-tuning function, which automatically calculates PID parameters by performing on-off oscillation. However, auto-tuning is a one-time process, and the parameters are fixed after tuning, making it impossible to continuously optimize during operation.

[0005] (3) Multi-channel temperature control module. Such as Omron EJ1, Siemens FM355 and other dedicated temperature control modules, which communicate with the main control PLC via bus. These modules are powerful but expensive, and each channel is controlled independently, making it impossible to achieve cross-channel collaborative optimization. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide an intelligent temperature control device and method based on an LSTM-DQN cascaded network to solve the problems in the background art.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] This application discloses an intelligent temperature control device based on an LSTM-DQN cascaded network, comprising:

[0009] The temperature acquisition module is used to acquire the historical temperature sequence of multiple temperature zones inside the vacuum oven during the current time period. The current time period includes multiple historical time points within the target duration before the current moment, and the historical temperature sequence includes the temperature values ​​of multiple historical time points.

[0010] The preprocessing module is used to preprocess the historical temperature sequence to obtain a preprocessed data sequence, wherein the preprocessing includes filtering, outlier removal, missing value filling and normalization.

[0011] The intelligent temperature control module is used to input the preprocessed data sequence into a pre-built LSTM-DQN cascaded temperature control model to obtain multiple temperature control adjustment actions, and to adjust the temperature of the temperature zone based on the temperature control adjustment actions. The LSTM-DQN cascaded temperature control model includes an LSTM temperature prediction model and a DQN temperature control model, and the output of the LSTM temperature prediction model is connected to the input of the DQN temperature control model.

[0012] In one embodiment of this application, the historical temperature sequence is preprocessed to obtain a preprocessed data sequence, including:

[0013] The historical temperature sequence is subjected to median filtering to obtain a filtered sequence;

[0014] The filtered sequence is subjected to outlier removal based on 3 times the standard deviation, and linear interpolation is performed on the missing positions to obtain the interpolated sequence;

[0015] The difference sequence is normalized to obtain a preprocessed data sequence.

[0016] In one embodiment of this application, the method for constructing the LSTM-DQN cascaded temperature control model includes:

[0017] Obtain historical temperature sequence samples from multiple temperature zones inside a vacuum oven;

[0018] Based on a pre-built sliding window, temperature sequence fragments of multiple time windows are extracted from the historical temperature sequence sample, and temperature sequence labels of a set duration after the end time of each time window are extracted. Metadata groups are constructed based on the temperature sequence fragments and temperature sequence labels of multiple time windows.

[0019] Multiple metadata groups are preprocessed to obtain multiple training samples;

[0020] The LSTM temperature prediction model is obtained by training the LSTM with multiple training samples.

[0021] A DQN training simulation environment was built based on the LSTM temperature prediction model, and the DQN network was trained in the DQN training simulation environment to obtain the DQN temperature control model.

[0022] The LSTM temperature prediction model and the DQN temperature control model are cascaded and jointly fine-tuned to obtain the LSTM-DQN cascaded temperature control model.

[0023] In one embodiment of this application, a DQN training simulation environment is built based on the LSTM temperature prediction model, including:

[0024] A simulation environment state update equation for a vacuum oven is established, wherein the mathematical expression of the simulation environment state update equation is:

[0025]

[0026] In the formula, For time index, This indicates the temperature at the current moment. Indicates the temperature at the next moment. Indicates the sampling period. This indicates the total heat capacity of the vacuum oven system. The rated power of the heating element, To control the input, For ambient temperature, For system thermal resistance;

[0027] A DQN training simulation environment is constructed based on the LSTM temperature prediction model and the simulation environment state update equation.

[0028] In one embodiment of this application, a DQN network is trained in a DQN training simulation environment to obtain a DQN temperature control model, including:

[0029] S1, set the target temperature and ambient temperature in the DQN training simulation environment, and obtain the current temperature at the current time step, wherein the current temperature is updated by the simulation environment state update equation of the previous time step, and the temperature of the first time step is the set ambient temperature.

[0030] S2, Construct the state of the current time step. The state This includes the current temperature, the set target temperature, the temperature change rate at the current time step, the predicted temperature at the current time step output by the LSTM temperature prediction model, and the cumulative temperature deviation.

[0031] S3, the state The input is fed into the DQN network to obtain the output action. The output action From a predefined action space Strategy selection;

[0032] S4, based on the output action The simulation environment state update equation updates the temperature and state at the next time step. The reward value at the current time step is calculated based on a pre-built reward function. The reward value The mathematical expression is:

[0033]

[0034] In the formula, , and All are weighted parameters. To set the target temperature, The temperature at the current time step. This represents the absolute value of the rate of temperature change. This is the overshoot indicator function;

[0035] S5, based on the state at the current time step Output Action Reward Value and the state at the next time step. Construct the experience vector for the current time step ;

[0036] S6, update the time step, and repeat steps S2-S5 until the temperature stabilizes beyond the target duration or reaches the maximum set number of steps, to obtain an empirical vector for multiple time steps;

[0037] S7. Randomly sample mini-batch empirical vectors from empirical vectors at multiple time steps, and train the DQN network based on the mini-batch empirical vectors and a pre-constructed loss function to obtain the DQN temperature control model.

[0038] In one embodiment of this application, a DQN network is trained based on the mini-batch empirical vectors and a pre-constructed loss function to obtain a DQN temperature control model, including:

[0039] S71, calculate the target Q value and the current Q value for each sample in the mini-batch experience vector, wherein the target Q value is calculated by the target network in the DQN network;

[0040] S72, calculate the loss between the target Q value and the current Q value based on the pre-constructed loss function, and adjust the main network in the DQN network based on the loss;

[0041] S73. Repeat steps S71-S72 until training is complete, and obtain the DQN temperature control model.

[0042] In one embodiment of this application, the LSTM temperature prediction model and the DQN temperature control model are cascaded and jointly fine-tuned to obtain an LSTM-DQN cascaded temperature control model, including:

[0043] The LSTM temperature prediction model and the DQN temperature control model are cascaded and deployed in the controller to obtain multiple real adjustment data.

[0044] The loss of multiple real moderated data points is calculated based on a pre-constructed joint loss function, wherein the mathematical expression of the joint loss function is:

[0045]

[0046]

[0047] In the formula, Indicates joint loss, This represents the weight parameters of the LSTM network. This represents the weight parameters of the DQN main network. A fine-tuning dataset constructed from multiple real-world adjustment data points. Indicates the fine-tuning of the dataset Experience samples from medium sampling The mathematical expectation, express The reward value at any moment, This is the discount factor. express Output actions at any time express The state at any given moment, express The state at any given moment, Indicates the action of maximizing the value. The Q value output by the target network. This represents the current Q-value output by the main network;

[0048] Based on the aforementioned joint loss, the cascaded LSTM temperature prediction model and DQN temperature control model are fine-tuned to obtain the LSTM-DQN cascaded temperature control model.

[0049] This application also provides an intelligent temperature control method based on LSTM-DQN cascaded networks, including:

[0050] Collect historical temperature sequences of multiple temperature zones inside a vacuum oven during the current time period, wherein the current time period includes multiple historical time points within the target duration prior to the current moment, and the historical temperature sequence includes temperature values ​​of multiple historical time points;

[0051] The historical temperature sequence is preprocessed to obtain a preprocessed data sequence, wherein the preprocessing includes filtering, outlier removal, missing value imputation, and normalization.

[0052] The preprocessed data sequence is input into a pre-built LSTM-DQN cascaded temperature control model to obtain multiple temperature control adjustment actions, and the temperature of the temperature zone is adjusted based on the temperature control adjustment actions. The LSTM-DQN cascaded temperature control model includes an LSTM temperature prediction model and a DQN temperature control model, and the output of the LSTM temperature prediction model is connected to the input of the DQN temperature control model.

[0053] This application also provides an electronic device, including: a processor and a memory;

[0054] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to cause the electronic device to perform the methods described above.

[0055] This application also provides a computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a processor, it implements the method described above.

[0056] The beneficial effects of this application are as follows: This application discloses an intelligent temperature control device and method based on an LSTM-DQN cascaded network. Through the prediction of future temperature trends by the LSTM network, the DQN network can respond in advance. Experimental results show that the control scheme of this invention can reduce the overshoot during the heating process from 15-20% of traditional PID to less than 5%, effectively avoiding material damage caused by excessive temperature. Through the design of a multi-objective reward function and the introduction of LSTM prediction information, this device can control steady-state temperature fluctuations within ±0.5℃, significantly better than the ±3-5℃ of traditional PID, meeting the requirements of high-precision applications such as lithium battery vacuum ovens. The continuous online learning capability of the DQN network enables the system to adapt to changes in operating conditions such as material batch variations, ambient temperature fluctuations, and heating element aging. Unlike traditional PID, which is tuned once and used for life, the control strategy of this invention is continuously optimized with the increase of the number of runs, resulting in continuous improvement in control accuracy. Attached Figure Description

[0057] The present application will be further described below with reference to the accompanying drawings and embodiments:

[0058] Figure 1 This is a schematic diagram of a three-layer architecture for an intelligent temperature control device shown in one embodiment of this application;

[0059] Figure 2 This is a schematic diagram of the LSTM network structure in one embodiment of this application;

[0060] Figure 3 This is a schematic diagram of the DQN network structure in one embodiment of this application;

[0061] Figure 4This is a structural diagram of an LSTM-DQN cascaded network in one embodiment of this application;

[0062] Figure 5 This is a schematic diagram of a multi-channel expansion scheme in one embodiment of this application;

[0063] Figure 6 This is a flowchart illustrating the online learning and model update process in one embodiment of this application;

[0064] Figure 7 This is a comparative experimental result diagram from this application;

[0065] Figure 8 This is a comparison curve of the control effect in this application. Detailed Implementation

[0066] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0067] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the layers related to this application and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0068] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of this application; however, it will be apparent to those skilled in the art that embodiments of this application may be practiced without these specific details.

[0069] The following table summarizes the prior art problems in this application:

[0070] Table 1. Summary of Existing Technical Problems:

[0071]

[0072] This invention aims to solve the aforementioned problems in the prior art and provides an intelligent temperature control device and method based on an LSTM-DQN cascaded network. The technical problems solved by this invention, the technical means employed, and the resulting technical effects form a complete correspondence, as shown in Table 2.

[0073] Table 2. Technical Problem Handling Methods and Technical Effects

[0074]

[0075] The core idea of ​​this invention is to construct a cascaded "prediction-decision" intelligent control architecture, in which the LSTM network is responsible for timing prediction and the DQN network is responsible for control decision-making. The two work together to achieve precise control of the heating equipment.

[0076] (1) Key technology association

[0077] This invention constructs a complete technology chain of "physical perception → data processing → intelligent decision-making → physical execution", as shown in Table 3.

[0078] Table 3 Key Technology Chain

[0079]

[0080] In this technology chain, each layer involves the interaction between physical entities and digital information, forming a complete closed loop of "physical → digital → physical". The algorithm does not exist independently, but is embedded in the technology system to solve the problem of "temperature control lag".

[0081] (2) Data flow direction and time sequence coordination

[0082] To ensure the realization of this invention, the following timing relationships are defined, as shown in Table 4.

[0083] Table 4 Timing Coordination Table

[0084]

[0085] In the above timing coordination, each step has a clear time sequence and data dependency relationship, which together constitute a complete control cycle (<100ms).

[0086] Figure 1 This is a schematic diagram of a three-layer architecture for an intelligent temperature control device shown in one embodiment of this application, as follows: Figure 1 As shown, the three-layer architecture of this application, from bottom to top, consists of: hardware layer, intelligent control layer, and application layer, as detailed below:

[0087] Hardware layer: Includes temperature sensors (PT100 or thermocouples), signal acquisition modules (analog inputs), edge computing cores (such as NVIDIA Jetson or high-performance PLCs), actuators (solid-state relays (SSRs) or thyristors), and the controlled object (heating plate / heating rod). Each temperature zone is configured with an independent sensor and actuator.

[0088] Intelligent Control Layer: The core of this invention is the LSTM-DQN cascaded network model deployed on the edge computing core. This layer is responsible for receiving real-time temperature data uploaded from the hardware layer, running the model for inference, outputting control commands, and continuously optimizing model parameters through an online learning module.

[0089] Application layer: This consists of a host computer or human-machine interface, providing functions such as parameter configuration, real-time monitoring, historical data query, and model management. Users can set target temperatures, select operating modes, and view control effects through the application layer.

[0090] The LSTM-DQN cascaded network model in this application adopts a cascaded structure of LSTM and DQN networks. The LSTM network, DQN network, and LSTM-DQN cascaded network are described below:

[0091] (1) LSTM network

[0092] LSTM (Long Short-Term Memory) is a special type of recurrent neural network (RNN) that can effectively learn long-term dependencies and is suitable for predicting temperature time series data. Figure 2 This is a schematic diagram of the LSTM network structure in one embodiment of this application, as shown below. Figure 2 As shown, the input layer of the LSTM network in this application receives temperature values ​​from the past N time steps, denoted as T=[t_{in},t_{i-n+1},...,t_{i-1}], where n is the time step. The hidden layer of the LSTM network uses LSTM units, with the number of hidden nodes set to H. The output layer of the LSTM network outputs the predicted temperature values ​​for the next M time steps, denoted as Ŷ=[ŷ_{i+1},ŷ_{i+2},...,ŷ_{i+M}], where M is the prediction step.

[0093] Table 5 shows the preferred parameter ranges for the LSTM network in this application. The preferred parameter ranges for the LSTM network are as shown in Table 5:

[0094] Table 5 Preferred Parameter Range Table

[0095]

[0096] (2) DQN decision network

[0097] DQN (Deep Q-Network) combines deep neural networks with Q-learning, enabling it to handle decision-making problems in high-dimensional state spaces and continuous action spaces.

[0098] (2-1) State-space design

[0099] In this application, the input state of the DQN decision network is defined as follows: , is represented as:

[0100]

[0101] In the formula, This indicates the temperature at the current moment. This indicates the set target temperature. The rate of temperature change (calculated by differentiation). The temperature prediction value output by the LSTM network. To accumulate temperature deviation, through the integral term calculate, The sampling period.

[0102] (2-2) Motion space design

[0103] action Defined as the heating output percentage, with a value range of [0, 100]. Preferably, it is discretized into 11 actions (0%, 10%, 20%, ..., 100%).

[0104] (2-3) Reward Function Design

[0105] The mathematical expression for the reward function is:

[0106]

[0107] In the formula, , and All are weighted parameters. To set the target temperature, The temperature at the current time step. This represents the absolute value of the rate of temperature change. This is the overshoot indicator function (set to 1 for overshoot, otherwise set to 0);

[0108] The preferred parameter ranges and their critical significance are shown in Table 6.

[0109] Table 6. Preferred Parameter Range

[0110]

[0111] Figure 3 This is a schematic diagram of the DQN network structure in one embodiment of this application, as shown below. Figure 3 As shown, the structure of the DQN network includes:

[0112] The input layer has 5 nodes (corresponding to the 5 dimensions of the state space).

[0113] Hidden layer 1, 64 nodes, ReLU activation function;

[0114] Hidden layer 2, 32 nodes, ReLU activation function;

[0115] The output layer has 11 nodes (corresponding to 11 discrete actions) and outputs the Q value of each action.

[0116] (3) LSTM-DQN cascaded network

[0117] Figure 4 This is a structural diagram of an LSTM-DQN cascaded network in one embodiment of this application, as shown below. Figure 4 As shown, this application uses the output of the LSTM network as the input of the DQN for cascading, thereby achieving coordination.

[0118] 1) Comparative experimental design

[0119] To verify that the cascading of LSTM and DQN produces a synergistic effect of "1+1>2", the present invention designed the following comparative experiment, which was tested under the same hardware platform and operating conditions, as shown in Table 7.

[0120] Table 7 Comparative Experiment Data Table

[0121]

[0122] 2) Analysis of Experimental Results

[0123] The overshoot of this invention (3.2%) was significantly lower than that of control group 1 (8.5%) and control group 2 (12.3%); the steady-state fluctuation of this invention (±0.3℃) was significantly better than that of control group 1 (±0.8℃) and control group 2 (±1.2℃); the adjustment time of this invention (8 minutes) was significantly better than that of control group 1 (12 minutes) and control group 2 (18 minutes); the energy saving effect of this invention (18.5%) was significantly better than that of control group 1 (12.3%) and control group 2 (8.7%).

[0124] 3) Quantification of synergistic effects

[0125] Compared to control group 1 (DQN only), the overshoot of the present invention is reduced by 62% and the energy saving effect is improved by 50%; compared to control group 2 (LSTM + fixed threshold only), the overshoot of the present invention is reduced by 74% and the energy saving effect is improved by 113%; indicating that the prediction information of LSTM and the decision-making ability of DQN have a synergistic enhancement effect, achieving an unexpected technical effect of "1+1>2".

[0126] The following technical advantages can be obtained through cascading collaboration, as shown in Table 8.

[0127] Table 8 Advantages of Cascaded Synergistic Technology

[0128]

[0129] 4) Multi-channel expansion scheme

[0130] This application supports independent control of multiple temperature zones, as shown in Table 9:

[0131] Table 9 Implementation methods for multi-temperature zone independent control

[0132]

[0133] Figure 5 This is a schematic diagram of a multi-channel expansion scheme in one embodiment of this application, showing a cascaded LSTM-DQN network with multi-channel expansion as follows: Figure 5 As shown.

[0134] Based on the aforementioned LSTM-DQN cascaded network structure, this application provides a detailed description of the training process for the temperature control model below. This application generally follows the steps of LSTM training, simulation environment setup, DQN model training, and joint fine-tuning, specifically including:

[0135] (1) LSTM training

[0136] The goal of this stage is to enable the LSTM to accurately predict future temperature trends based on historical temperature sequences. Specifically, this includes:

[0137] (1-1) Obtain historical temperature sequence samples of multiple temperature zones inside the vacuum oven;

[0138] The actual temperature control device operates using traditional PID control or manual control, collecting baking data over a long period. The data should include: timestamp, measured temperature, heating power (control output), and target temperature. The sampling period is 100ms.

[0139] (1-2) Extract temperature sequence fragments of multiple time windows from the historical temperature sequence sample based on a pre-built sliding window, extract temperature sequence labels after a set duration following the end time of each time window, and construct metadata groups based on the temperature sequence fragments and temperature sequence labels of multiple time windows.

[0140] The sliding window method is used. The input feature X is the temperature value of the past n times (e.g., n corresponds to 30-120 minutes, i.e., 18000-72000 time points; in practice, downsampling can be performed, such as taking the average per second, and converting the time length corresponding to the window into a reasonable number of steps). The output label Y is the temperature value of the next M times (e.g., the temperature 5-30 minutes from now).

[0141] (1-3) Preprocess multiple metadata groups to obtain multiple training samples;

[0142] Preprocessing includes filtering, outlier removal, missing value interpolation, and normalization.

[0143] (1-4) The LSTM is trained based on multiple training samples to obtain the LSTM temperature prediction model;

[0144] In this application, the mean squared error (MSE) is used as the loss function, and the Adam optimizer is used for training. After training, the LSTM network can predict future temperature change trends based on the current temperature sequence.

[0145] (2) Simulation environment setup

[0146] In this stage, the LSTM network acts as an environmental "predictor," providing enhanced state information to the DQN. The DQN learns the optimal policy by interacting with the environment (an actual temperature control device or a high-precision simulation model). Specifically,

[0147] To safely and efficiently pre-train a DQN, a simulator of a temperature control system can be built using historical data or a thermodynamic model. The simulator's input is the heating power (i.e., the action). The output is temperature change. In this simulator, the predictions of the LSTM can also be calculated, thus fully simulating the operation of the cascaded model.

[0148] (2-1) Establish the simulation environment state update equation (i.e., simulator) for the vacuum oven, wherein the mathematical expression of the simulation environment state update equation is:

[0149]

[0150] In the formula, For time index, This indicates the temperature at the current moment. Indicates the temperature at the next moment. Indicates the sampling period. This indicates the total heat capacity of the vacuum oven system. The rated power of the heating element, To control the input, For ambient temperature, For system thermal resistance;

[0151] (2-2) Construct a DQN training simulation environment based on the LSTM temperature prediction model and the simulation environment state update equation.

[0152] (3) DQN model training

[0153] S1, set the target temperature and ambient temperature in the DQN training simulation environment, and obtain the current temperature at the current time step, wherein the current temperature is updated by the simulation environment state update equation of the previous time step, and the temperature of the first time step is the set ambient temperature.

[0154] S2, Construct the state of the current time step. The state This includes the current temperature, the set target temperature, the temperature change rate at the current time step, the predicted temperature at the current time step output by the LSTM temperature prediction model, and the cumulative temperature deviation.

[0155]

[0156] S3, the state The input is fed into the DQN network to obtain the output action. The output action From a predefined action space Strategy selection;

[0157] Specifically, a discrete action space is defined (as described above), and then random numbers are generated. ,like If the condition is met, then an action is randomly selected from the discrete action space. Otherwise, the state is... Input the data into the DQN main network, calculate the Q-value for each action, and select the action with the largest Q-value.

[0158] Specifically, in the early stages of training Set it to a high value (e.g., 0.9), and gradually reduce it to a minimum value (e.g., 0.01) with the number of training steps using a decay factor (e.g., 0.999).

[0159] It balances exploration (trying unknown actions) with utilization (adopting the current best action). The high exploration rate in the early stage helps the agent discover the effects of different heating strategies; the low exploration rate in the later stage ensures stable control.

[0160] S4, based on the output action The simulation environment state update equation updates the temperature and state at the next time step. The reward value at the current time step is calculated based on a pre-built reward function. The reward value The mathematical expression is:

[0161]

[0162] In the formula, , and All are weighted parameters. To set the target temperature, The temperature at the current time step. This represents the absolute value of the rate of temperature change. This is the overshoot indicator function;

[0163] The reward function simultaneously penalizes rapid temperature rise during temperature deviation, high energy consumption, and overshoot, guiding DQN to learn a comprehensive optimal strategy that achieves high accuracy, low energy consumption, and small overshoot.

[0164] S5, based on the state at the current time step Output Action Reward Value and the state at the next time step. Construct the experience vector for the current time step ;

[0165] Specifically, the experience pool adopts a circular queue structure with a preset capacity (e.g., 10,000), and new data overwrites old data. The experience replay mechanism breaks the temporal correlation between consecutive samplings, making the training samples independent and identically distributed, thus improving learning stability.

[0166] S6, update the time step, and repeat steps S2-S5 until the temperature stabilizes beyond the target duration or reaches the maximum set number of steps, to obtain an empirical vector for multiple time steps;

[0167] For each step executed, the time step index... Repeat steps S2-S5. The round terminates when: ① the temperature remains stable for more than a preset duration (e.g., 5 minutes, meaning the absolute value of temperature change is less than 0.1℃ / s for 3000 consecutive steps); ② the total number of steps reaches the maximum set value (e.g., 36000 steps, corresponding to 1 hour). Upon reaching the termination condition, save all experience vectors collected in the current round and begin a new round (resetting the environment or changing the target temperature).

[0168] A complete cycle can cover the entire process of heating up, approaching the set value, overshoot, and steady-state adjustment, providing diverse training samples. Termination conditions prevent infinite running and ensure training progress.

[0169] S7. Randomly sample mini-batch empirical vectors from empirical vectors at multiple time steps, and train the DQN network based on the mini-batch empirical vectors and a pre-constructed loss function to obtain the DQN temperature control model.

[0170] S71, calculate the target Q value and the current Q value for each sample in the mini-batch experience vector, wherein the target Q value is calculated by the target network in the DQN network;

[0171] S72, calculate the loss between the target Q value and the current Q value based on the pre-constructed loss function, and adjust the main network in the DQN network based on the loss;

[0172] The loss function uses the mean squared error function, and its expression is:

[0173]

[0174] In the formula, Indicates loss, Indicates the main network parameters. Indicates the small batch size. For sample index, For the target Q value, The current Q value, Indicates sample state, Indicates sample The action.

[0175] The loss function calculates the squared difference between the "target value" and the "predicted value" for each sample, and then takes a mini-batch average. By minimizing this loss, the output of the main network... It will gradually approach a more accurate In this way, the optimal action value function can be learned.

[0176] S73. Repeat steps S71-S72 until training is complete, and obtain the DQN temperature control model.

[0177] (4) Joint fine-tuning

[0178] (4-1) The LSTM temperature prediction model and the DQN temperature control model are cascaded and deployed in the controller to obtain multiple real adjustment data.

[0179] Specifically, the simulated LSTM temperature prediction model (frozen or to be fine-tuned) is cascaded with the DQN temperature control model (main network) and deployed to the edge controller (such as NVIDIA Jetson) of a real vacuum oven. The target temperature is then set. (For example, 150°C), start the temperature control device and run the complete baking process (from room temperature to the set value and into a steady state).

[0180] The entire operation process data is recorded at a fixed control cycle (e.g., 100ms). Each record includes: timestamp, status. (Current temperature, target temperature, rate of temperature change, LSTM predicted temperature, cumulative deviation), dynamic (Heating power percentage), Rewards (Calculated in real time based on the reward function), Next State .

[0181] Repeatedly run multiple batches (e.g., 3-5 different target temperatures or different load conditions) to collect thousands to tens of thousands of continuous empirical samples, forming a fine-tuning dataset. .

[0182] (4-2) Calculate the loss of multiple real adjustment data based on a pre-constructed joint loss function, wherein the mathematical expression of the joint loss function is:

[0183]

[0184]

[0185] In the formula, Indicates joint loss, This represents the weight parameters of the LSTM network. This represents the weight parameters of the DQN main network. A fine-tuning dataset constructed from multiple real-world adjustment data points. Indicates the fine-tuning of the dataset Experience samples from medium sampling The mathematical expectation, express The reward value at any moment, This is the discount factor. express Output actions at any time express The state at any given moment, express The state at any given moment, Indicates the action of maximizing the value. The Q value output by the target network. This represents the current Q-value output by the main network;

[0186] The joint loss function is essentially still the TD error of DQN, but the prediction results of LSTM are involved in the state. and The construction. Therefore, when the loss is... When differentiating, the gradient will also pass through... Backpropagation to This enables end-to-end optimization.

[0187] Unlike the MSE loss used in the pre-training phase of LSTM, the joint loss forces LSTM to learn to generate the predictive features that are most helpful for DQN decision-making (rather than simply minimizing the temperature prediction error), thereby achieving a synergistic enhancement of "prediction-decision".

[0188] (4-3) Based on the joint loss, the cascaded LSTM temperature prediction model and DQN temperature control model are fine-tuned to obtain the LSTM-DQN cascaded temperature control model.

[0189] Joint fine-tuning is a combination of transfer learning and end-to-end optimization: using pre-trained LSTM and DQN as initialization, and making small adjustments to them with a small amount of real data, so that the model can quickly adapt to the characteristics of the target domain (real temperature control system).

[0190] The extremely low learning rate ensures that the fine-tuning process will not result in "catastrophic forgetting," that is, it will not destroy the good control foundation learned in the simulation environment.

[0191] By using joint gradient propagation, LSTM networks can learn "what predictions can help DQN estimate future returns more accurately," thus overcoming the limitations of independent predictions.

[0192] In addition, this application provides an online learning mechanism, which can be set to trigger at a set time, once after each baking batch is completed (preferably every 4-8 hours); trigger due to an anomaly, when a temperature fluctuation is detected to exceed a threshold (e.g., ±1℃); and trigger manually by the user through a host computer interface. Figure 6 This is a flowchart illustrating the online learning and model update process in one embodiment of this application. The online learning and model update process is as follows: Figure 6 As shown. Specifically, temperature data and control records from the most recent operating cycle are collected; the DQN network is fine-tuned using the new data (learning rate reduced to 1 / 10 of the initial value); optionally, the LSTM network is retrained (when temperature characteristics change significantly); the performance of the new model is verified on the validation set; if the new model improves performance (e.g., the reward function value increases by more than 5%), the current model is replaced; otherwise, the original model is retained. Progressive learning is adopted to avoid control instability caused by sudden model changes; a backup model is retained, and the new model is replaced only after successful verification; the learning rate decay strategy is as follows: the initial learning rate lr_initial=0.001, which decays to 0.9 times the original value after every 10 online learning cycles.

[0193] The specific implementation method is as follows:

[0194] Based on the NVIDIA Jetson 8-channel lithium-ion vacuum oven intelligent temperature control system, taking a certain model of lithium-ion vacuum oven as an example, the specific implementation of the present invention is described in detail.

[0195] 1. Hardware Configuration

[0196] Table 10 Hardware Configuration Table

[0197]

[0198] 2. Software Implementation Steps

[0199] Step 1: Data Acquisition and Preprocessing (every 100ms)

[0200] The data acquisition module uploads temperature data from eight temperature zones to the PLC via the Modbus RTU protocol;

[0201] The PLC collects temperature data every 100ms and sends the data to Jetson via ModbusTCP; Jetson stores the temperature data in a time-series database (such as InfluxDB) for model training.

[0202] Data preprocessing: The temperature data is normalized using the formula T_norm=(T-T_min) / (T_max-T_min), where T_min=0℃ and T_max=200℃.

[0203] Step 2: Training the LSTM prediction network

[0204] Collected 30 days of historical baking data, totaling approximately 26 million temperature records;

[0205] Construct training samples: Each sample contains temperature values ​​for the first 60 time points (60 minutes) and is labeled with the temperature values ​​for the last 15 time points (15 minutes);

[0206] Build an LSTM network using a Python deep learning framework (such as TensorFlow or PyTorch): 60 nodes in the input layer, 128 hidden units in the LSTM layer, and 15 nodes in the output layer;

[0207] Training parameters: batchsize=32, epochs=100, learning rate=0.001, loss function=MSE;

[0208] After training, export the model in ONNX format and deploy it to Jetson.

[0209] Step 3: Construction and Training of DQN Decision Network

[0210] Define the state space, define the action space, define the reward function, and construct the DQN network (input layer: 5 nodes; hidden layer 1: 64 nodes, ReLU activation; hidden layer 2: 32 nodes, ReLU activation; output layer: 11 nodes).

[0211] Training parameters: Experience pool capacity = 10000, target network update frequency = 100 steps, initial exploration rate ε = 0.9, exploration rate decay coefficient = 0.995, minimum exploration rate ε_min = 0.05, learning rate = 0.001, batch size = 32.

[0212] Step 4: Deploy cascaded models (execute every 100ms)

[0213] 1) Read the current temperature T_current from the PLC;

[0214] 2) Retrieve the first 60 temperature values ​​from the historical cache;

[0215] 3) Use the LSTM network for inference to obtain T_lstm_pred (the predicted temperature 15 minutes later);

[0216] 4) Calculate dT / dt and cumulative error;

[0217] 5) Construct the state vector S_t;

[0218] 6) DQN network inference, select the optimal action a_t (output percentage);

[0219] 7) Convert a_t to PWM duty cycle and write it to the PLC via ModbusTCP;

[0220] 8) The PLC controls the SSR output to achieve heating control of the heating plate.

[0221] Step 5: Online Learning and Model Updates

[0222] Each completed baking batch (approximately 4 hours) triggers an online learning session:

[0223] 1) Extract the temperature data and control records for this batch from the database;

[0224] 2) Fine-tune the DQN network using new data (reducing the learning rate to 0.0001);

[0225] 3) Validate the performance of the new model on the validation set (the validation set is the last 20% of the data in this batch).

[0226] 4) If the average reward value of the new model increases by more than 5%, then replace the current model; otherwise, retain the original model.

[0227] The LSTM network is retrained monthly to adapt to long-term changes.

[0228] 3. Operation Results and Verification

[0229] This system has been applied to the vacuum oven of a lithium battery manufacturer, enabling independent temperature control for eight temperature zones in the upper and lower molds. The test results are shown in Table 11. Figure 6 This is a comparative experimental result diagram from this application. Figure 7 The control effect comparison curve in this application is shown below, comparing the experimental effect graph and the control effect comparison curve, such as... Figure 6 and Figure 7 As shown.

[0230] Table 11 Test Results

[0231]

[0232] 1) Fault simulation verification

[0233] If a thermocouple in a certain temperature zone is manually disconnected, the system will detect the temperature abnormality within 200ms, force the DQN output to zero, stop heating in the corresponding channel, and trigger an alarm on the HMI.

[0234] The system simulates an SSR breakdown fault, detects the anomaly through current feedback, sets a fault flag, and prompts maintenance.

[0235] 2) Model convergence verification

[0236] After 10 online learning sessions with different baking batches, the average reward value of the DQN network improved from an initial -50 to -25, a 50% increase.

[0237] After 50 batches, the reward value stabilized at around -22, indicating that the model had converged.

[0238] 3) Comparative experimental verification

[0239] Table 12 Comparative Experiment Table

[0240]

[0241] 4) Quantification of synergistic effects

[0242] Compared to control group 1 (DQN only), the overshoot was reduced by 62% and the energy saving effect was improved by 50%;

[0243] Compared to control group 2 (LSTM only + fixed threshold), the overshoot was reduced by 74% and the energy saving effect was improved by 113%.

[0244] After training is completed, an intelligent temperature control device based on an LSTM-DQN cascaded network, based on the above LSTM-DQN control model, includes:

[0245] The temperature acquisition module is used to acquire the historical temperature sequence of multiple temperature zones inside the vacuum oven during the current time period. The current time period includes multiple historical time points within the target duration before the current moment, and the historical temperature sequence includes the temperature values ​​of multiple historical time points.

[0246] The preprocessing module is used to preprocess the historical temperature sequence to obtain a preprocessed data sequence, wherein the preprocessing includes filtering, outlier removal, missing value filling and normalization.

[0247] The intelligent temperature control module is used to input the preprocessed data sequence into a pre-built LSTM-DQN cascaded temperature control model to obtain multiple temperature control adjustment actions, and to adjust the temperature of the temperature zone based on the temperature control adjustment actions. The LSTM-DQN cascaded temperature control model includes an LSTM temperature prediction model and a DQN temperature control model, and the output of the LSTM temperature prediction model is connected to the input of the DQN temperature control model.

[0248] The above preprocessing process includes:

[0249] The historical temperature sequence is subjected to median filtering to obtain a filtered sequence;

[0250] The filtered sequence is subjected to outlier removal based on 3 times the standard deviation, and linear interpolation is performed on the missing positions to obtain the interpolated sequence;

[0251] The difference sequence is normalized to obtain a preprocessed data sequence.

[0252] In addition, this application also provides an intelligent temperature control method based on LSTM-DQN cascaded networks, including:

[0253] Collect historical temperature sequences of multiple temperature zones inside a vacuum oven during the current time period, wherein the current time period includes multiple historical time points within the target duration prior to the current moment, and the historical temperature sequence includes temperature values ​​of multiple historical time points;

[0254] The historical temperature sequence is preprocessed to obtain a preprocessed data sequence, wherein the preprocessing includes filtering, outlier removal, missing value imputation, and normalization.

[0255] The preprocessed data sequence is input into a pre-built LSTM-DQN cascaded temperature control model to obtain multiple temperature control adjustment actions, and the temperature of the temperature zone is adjusted based on the temperature control adjustment actions. The LSTM-DQN cascaded temperature control model includes an LSTM temperature prediction model and a DQN temperature control model, and the output of the LSTM temperature prediction model is connected to the input of the DQN temperature control model.

[0256] This application discloses an intelligent temperature control device and method based on an LSTM-DQN cascaded network. By using the LSTM network to predict future temperature trends, the DQN network can respond in advance. Experimental results show that the control scheme of this invention can reduce the overshoot during the heating process from 15-20% in traditional PID control to less than 5%, effectively avoiding material damage caused by excessive temperature. Through the design of a multi-objective reward function and the introduction of LSTM prediction information, this device can control steady-state temperature fluctuations within ±0.5℃, significantly better than the ±3-5℃ of traditional PID control, meeting the requirements of high-precision applications such as lithium battery vacuum ovens. The continuous online learning capability of the DQN network enables the system to adapt to changes in operating conditions such as material batch variations, ambient temperature fluctuations, and heating element aging. Unlike traditional PID control, which is tuned once and used for life, the control strategy of this invention is continuously optimized with the increase of the number of runs, resulting in continuous improvement in control accuracy.

[0257] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0258] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0259] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0260] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0261] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0262] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0263] In the above embodiments, although the present application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present application are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0264] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A smart temperature control device based on an LSTM-DQN cascaded network, characterized in that, include: The temperature acquisition module is used to acquire the historical temperature sequence of multiple temperature zones inside the vacuum oven during the current time period. The current time period includes multiple historical time points within the target duration before the current moment, and the historical temperature sequence includes the temperature values ​​of multiple historical time points. The preprocessing module is used to preprocess the historical temperature sequence to obtain a preprocessed data sequence, wherein the preprocessing includes filtering, outlier removal, missing value filling and normalization. The intelligent temperature control module is used to input the preprocessed data sequence into a pre-built LSTM-DQN cascaded temperature control model to obtain multiple temperature control adjustment actions, and to adjust the temperature of the temperature zone based on the temperature control adjustment actions. The LSTM-DQN cascaded temperature control model includes an LSTM temperature prediction model and a DQN temperature control model, and the output of the LSTM temperature prediction model is connected to the input of the DQN temperature control model.

2. The intelligent temperature control device based on an LSTM-DQN cascaded network according to claim 1, characterized in that, The historical temperature sequence is preprocessed to obtain a preprocessed data sequence, including: The historical temperature sequence is subjected to median filtering to obtain a filtered sequence; The filtered sequence is subjected to outlier removal based on 3 times the standard deviation, and linear interpolation is performed on the missing positions to obtain the interpolated sequence; The difference sequence is normalized to obtain a preprocessed data sequence.

3. The intelligent temperature control device based on an LSTM-DQN cascaded network according to claim 1, characterized in that, The method for constructing the LSTM-DQN cascaded temperature control model includes: Obtain historical temperature sequence samples from multiple temperature zones inside a vacuum oven; Based on a pre-built sliding window, temperature sequence fragments of multiple time windows are extracted from the historical temperature sequence sample, and temperature sequence labels of a set duration after the end time of each time window are extracted. Metadata groups are constructed based on the temperature sequence fragments and temperature sequence labels of multiple time windows. Multiple metadata groups are preprocessed to obtain multiple training samples; The LSTM temperature prediction model is obtained by training the LSTM with multiple training samples. A DQN training simulation environment was built based on the LSTM temperature prediction model, and the DQN network was trained in the DQN training simulation environment to obtain the DQN temperature control model. The LSTM temperature prediction model and the DQN temperature control model are cascaded and jointly fine-tuned to obtain the LSTM-DQN cascaded temperature control model.

4. The intelligent temperature control device based on an LSTM-DQN cascaded network according to claim 3, characterized in that, A DQN training and simulation environment was built based on the LSTM temperature prediction model, including: A simulation environment state update equation for a vacuum oven is established, wherein the mathematical expression of the simulation environment state update equation is: In the formula, For time index, This indicates the temperature at the current moment. Indicates the temperature at the next moment. Indicates the sampling period. This indicates the total heat capacity of the vacuum oven system. The rated power of the heating element, To control the input, For ambient temperature, For system thermal resistance; A DQN training simulation environment is constructed based on the LSTM temperature prediction model and the simulation environment state update equation.

5. The intelligent temperature control device based on an LSTM-DQN cascaded network according to claim 4, characterized in that, The DQN network was trained in a DQN training simulation environment to obtain a DQN temperature control model, including: S1, in the DQN training simulation environment, set the target temperature and ambient temperature, and obtain the current temperature at the current time step, wherein the current temperature is updated by the simulation environment state update equation of the previous time step, and the temperature of the first time step is the set ambient temperature. S2, Construct the state of the current time step. The state This includes the current temperature, the set target temperature, the temperature change rate at the current time step, the predicted temperature at the current time step output by the LSTM temperature prediction model, and the cumulative temperature deviation. S3, the state The input is fed into the DQN network to obtain the output action. The output action From a predefined action space Strategy selection; S4, based on the output action The simulation environment state update equation updates the temperature and state at the next time step. The reward value at the current time step is calculated based on a pre-built reward function. The reward value The mathematical expression is: In the formula, , and All are weighted parameters. To set the target temperature, The temperature at the current time step. This represents the absolute value of the rate of temperature change. This is the overshoot indicator function; S5, based on the state at the current time step Output Action Reward Value And the state at the next time step. Construct the experience vector for the current time step ; S6, update the time step, and repeat steps S2-S5 until the temperature stabilizes beyond the target duration or reaches the maximum set number of steps, to obtain an empirical vector for multiple time steps; S7. Randomly sample mini-batch empirical vectors from empirical vectors at multiple time steps, and train the DQN network based on the mini-batch empirical vectors and a pre-built loss function to obtain the DQN temperature control model.

6. The intelligent temperature control device based on an LSTM-DQN cascaded network according to claim 5, characterized in that, The DQN network is trained based on the mini-batch empirical vectors and a pre-constructed loss function to obtain the DQN temperature control model, including: S71, calculate the target Q value and the current Q value for each sample in the mini-batch experience vector, wherein the target Q value is calculated by the target network in the DQN network; S72, calculate the loss between the target Q value and the current Q value based on the pre-constructed loss function, and adjust the main network in the DQN network based on the loss; S73. Repeat steps S71-S72 until training is complete, and obtain the DQN temperature control model.

7. The intelligent temperature control device based on an LSTM-DQN cascaded network according to claim 3, characterized in that, The LSTM temperature prediction model and the DQN temperature control model are cascaded and jointly fine-tuned to obtain the LSTM-DQN cascaded temperature control model, including: The LSTM temperature prediction model and the DQN temperature control model are cascaded and deployed in the controller to obtain multiple real adjustment data. The loss of multiple real moderated data points is calculated based on a pre-constructed joint loss function, wherein the mathematical expression of the joint loss function is: In the formula, Indicates joint loss, This represents the weight parameters of the LSTM network. This represents the weight parameters of the DQN main network. A fine-tuning dataset constructed from multiple real-world adjustment data points. Indicates the fine-tuning of the dataset Experience samples from medium sampling The mathematical expectation, express The reward value at any moment, This is the discount factor. express Output actions at any time express The state at any given moment, express The state at any given moment, Indicates the action of maximizing the value. The Q value output by the target network. This represents the current Q-value output by the main network; Based on the aforementioned joint loss, the cascaded LSTM temperature prediction model and DQN temperature control model are fine-tuned to obtain the LSTM-DQN cascaded temperature control model.

8. A smart temperature control method based on LSTM-DQN cascaded networks, characterized in that, include: Collect historical temperature sequences of multiple temperature zones inside a vacuum oven during the current time period, wherein the current time period includes multiple historical time points within the target duration prior to the current moment, and the historical temperature sequence includes temperature values ​​of multiple historical time points; The historical temperature sequence is preprocessed to obtain a preprocessed data sequence, wherein the preprocessing includes filtering, outlier removal, missing value imputation, and normalization. The preprocessed data sequence is input into a pre-built LSTM-DQN cascaded temperature control model to obtain multiple temperature control adjustment actions, and the temperature of the temperature zone is adjusted based on the temperature control adjustment actions. The LSTM-DQN cascaded temperature control model includes an LSTM temperature prediction model and a DQN temperature control model, and the output of the LSTM temperature prediction model is connected to the input of the DQN temperature control model.

9. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images