Vacuum furnace heating strategy dynamic adjustment method based on reinforcement learning driving

Abstract

The invention discloses a vacuum furnace heating strategy dynamic adjustment method based on reinforcement learning driving, and belongs to the technical field of vacuum furnace heating control. The method comprises the following steps: acquiring characteristic data of the vacuum furnace and constructing a dynamic simulation model of the vacuum furnace; constructing an intelligent agent comprising a state sensing module, a strategy network, a value network, an experience playback buffer area, a parameter updating unit, an action execution unit and a reward receiving unit, putting the intelligent agent into the vacuum furnace dynamic simulation model, and performing multi-step interactive training to obtain a dynamic heating control model; integrating the dynamic heating control model into control software of the vacuum furnace, monitoring running state data in real time when the vacuum furnace runs, and judging whether to retrain the dynamic heating control model or not based on a preset performance threshold value. The heating strategy is driven to be dynamically adjusted through reinforcement learning, the temperature control precision can be improved, energy consumption can be optimized, the adaptability and stability of the heating process of the vacuum furnace are enhanced, and the heating effect of the vacuum furnace is improved.

Description

Technical Field

[0001] This invention relates to the field of vacuum furnace heating control technology, and specifically to a method for dynamically adjusting vacuum furnace heating strategies based on reinforcement learning. Background Technology

[0002] In the field of vacuum furnace heating control, the research and application of traditional heating strategies have long relied on physical experimental debugging or offline parameter tuning based on simplified models. The former requires repeated heating tests on a physical vacuum furnace, which not only occupies equipment production time and increases material consumption, but also poses operational risks in high-temperature environments and is difficult to fully cover different operating conditions; the latter, because the simplified model cannot accurately reflect the complex heat transfer laws in the furnace, results in significant deviations in the offline-tuned control parameters in practical applications, requiring frequent rework and adjustment.

[0003] Meanwhile, existing control strategies based on fixed algorithms, while capable of basic heating control, have significant shortcomings in dynamic adaptability: on the one hand, in actual operation of vacuum furnaces, equipment characteristics may drift, and fixed parameter algorithms are difficult to correct in real time, which can easily lead to problems such as temperature overshoot and heat preservation fluctuations; on the other hand, in the face of the demand for multi-variety and small-batch production, a large number of parameters of the vacuum furnace need to be manually reconfigured, which is not only inefficient but also relies on the experience of operators and makes it difficult to ensure control consistency.

[0004] With the development of industrial intelligence, higher demands are placed on the precision, energy efficiency, and adaptability of vacuum furnace heating processes. Traditional control modes relying on physical experiments and fixed algorithms have become bottlenecks restricting production efficiency and product quality. Therefore, there is an urgent need for an intelligent control method that can dynamically optimize heating strategies, adapt quickly, and accommodate complex operating conditions. To this end, a reinforcement learning-driven dynamic adjustment method for vacuum furnace heating strategies is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning includes the following steps: S1. Obtain vacuum furnace characteristic data and construct a dynamic simulation model of the vacuum furnace based on the vacuum furnace characteristic data; S2. Construct an intelligent agent, put the intelligent agent into the dynamic simulation model of the vacuum furnace for simulation training, and obtain a dynamic heating control model; S3. Integrate the dynamic heating control model into the control software of the vacuum furnace. The vacuum furnace operates by running the control software with the integrated dynamic heating control model and monitors the operating status data of the vacuum furnace in real time to determine whether the dynamic heating control model needs to be re-simulated and trained.

[0007] Preferably, the vacuum furnace characteristic data are core parameters reflecting the physical structure, operating mechanism, and heating characteristics of the vacuum furnace. Specifically, these include heating system parameters, furnace chamber structure parameters, vacuum system parameters, control system parameters, heat transfer parameters, and material thermal properties. Heating system parameters cover the type, rated power, quantity, and distribution of heating elements, voltage and current parameters of the heating circuit, and the thermal response curves of the heating elements. Furnace chamber structure parameters include the furnace chamber's geometry, internal cavity shape, insulation material and thickness, and furnace door sealing method, used to accurately simulate the spatial distribution and heat conduction boundary conditions within the furnace chamber. Vacuum system parameters involve the type of vacuum pump, pumping rate, and ultimate vacuum level, as well as the vacuum valve... The control logic of the gate, as well as the adjustment range of pressure and gas composition in the furnace, affect the heat transfer efficiency in the furnace. The control system parameters include the response delay time of the original control module of the vacuum furnace, the action accuracy of the actuator, and the measurement range and error range of the sensor, to ensure that the simulation model can reflect the execution deviation of the actual control command. The heat transfer parameters cover the thermal radiation coefficient, convective heat transfer coefficient, and solid thermal conductivity in the furnace, which are used to accurately calculate the heat transfer path and rate in the furnace. The thermal properties of the material include the specific heat capacity, thermal conductivity, melting point, and latent heat of phase change of the material to be heated. The differences in the thermal properties of different materials will directly affect the temperature change law during the heating process, which is the key basis for the simulation model to adapt to various heating tasks.

[0008] Preferably, the method for constructing a dynamic simulation model of a vacuum furnace based on vacuum furnace characteristic data is as follows: Based on the characteristic data of the vacuum furnace, a dynamic simulation model of the vacuum furnace is constructed using computer simulation technology. The specific process is as follows: a three-dimensional geometric model of the vacuum furnace chamber is established based on the furnace chamber structural parameters to restore the internal spatial structure of the furnace chamber and the actual layout of heating elements and sensors; based on heat transfer parameters and vacuum system parameters, the heat transfer mode and vacuum environment boundary conditions of the three-dimensional geometric model of the vacuum furnace chamber are defined; at the same time, based on the heating system parameters and control system parameters, the mapping relationship between heating power output and control commands is constructed to simulate the dynamic response of the actuator, and the thermal properties of the material are embedded so that the three-dimensional geometric model of the vacuum furnace chamber can automatically adjust the thermal response law according to different heating materials.

[0009] Preferably, the intelligent agent includes a state perception module, a policy network, a value network, an experience replay buffer, a parameter update unit, an action execution unit, and a reward receiving unit; The state perception module is used to acquire state information in real time from the data interface of the vacuum furnace dynamic simulation model, and then normalize the state information and convert it into an 8-dimensional state vector. ; The policy network is used to express state vectors. As input, it is processed through a multi-layer neural network, and the output is a state vector. The corresponding heating action probability distribution uses a multilayer perceptron (MLP) for the multilayer neural network, which includes an 8-dimensional input layer, a first hidden layer with 128 neurons, a second hidden layer with 64 neurons, and an output layer. The value network is used to receive state vectors. Output state vector Corresponding state value estimate ; The experience replay buffer is used to receive the state vector. and Heating action and action rewards and will receive the state vector and Heating action and action rewards Integrating into quadruple samples Store, where The vacuum furnace dynamic simulation model executes the heating action. The next state vector; The parameter update unit is used to obtain the quadruplet samples stored in the experience replay buffer. and value network output Based on this, the loss function of the policy network is calculated. and the loss function of the value network And according to the loss function of the policy network and the loss function of the value network Minimize the loss function of the policy network using the gradient descent algorithm. and the loss function of the value network And update the parameters of the policy network and the value network; The formula for the loss function of the computational value network is: ; in, It is the number of quadruplets. It is the state value estimate of the i-th quadruple sample from the value network. It is the target value of the i-th quadruple sample; The formula for the loss function of the computational strategy network is: ; in, It is the cutoff coefficient. It is the strategy ratio of the i-th quadruple sample. It is the advantage function value of the i-th quadruple sample and , It is the number of quadruplets. It means to Limited to Within the range; The action execution unit is used to select the action with the highest probability as the heating action based on the action probability distribution using a greedy strategy. and heating action Send to the vacuum furnace dynamic simulation model for execution; The reward receiving unit is used to obtain the dynamic simulation model of the vacuum furnace when performing heating actions. Post-action reward .

[0010] Preferably, the reward receiving unit acquires the dynamic simulation model of the vacuum furnace during the execution of heating actions. Post-action reward The process: The dynamic simulation model of the vacuum furnace executes the heating action. Then, new state information is generated. The reward receiving unit receives the new state information and uses it to calculate the action reward using the reward function formula. The new status information includes heating power. Furnace pressure Gas flow rate and monitoring point temperature ; The reward function formula is as follows: ; in, , , , and These are weighting coefficients set according to the process objectives of the vacuum furnace. It is the difference between the furnace pressure and the target process furnace pressure. It is the difference between the gas flow rate and the target process gas flow rate. Temperature at the monitoring point The difference between the mean and the target process temperature. Temperature at the monitoring point The average value and the temperature at the monitoring point at the previous moment The difference between the means; The action reward It measures the heating action Quantitative indicators of merits and demerits: like A high positive value indicates that heating is in operation. This allows for closer temperature control to the target, lower energy consumption, and more stable processes. like A negative or low value indicates that heating is in operation. There are issues such as excessive temperature deviation, high energy consumption, and insufficient stability. The data will be fed into the experience replay buffer, providing crucial information for subsequent parameter update units to optimize the parameters of the policy network and value network, thus driving the agent to gradually learn a better heating strategy.

[0011] Preferably, the method of placing the intelligent agent into a dynamic simulation model of a vacuum furnace for simulation training is as follows: S01. Initialize the parameters of the agent's policy network and value network, and initialize the vacuum furnace dynamic simulation model, and set the initial state information of the vacuum furnace dynamic simulation model. S02. The state awareness module acquires initial state information in real time from the dynamic simulation model of the vacuum furnace and outputs a state vector. ; S03. The policy network uses state vectors. Input is the state vector, output is the state vector. The corresponding action probability distribution, and the value network receives the state vector. Output state vector Corresponding state value estimate ; S04. The action execution unit selects the heating action based on the action probability distribution. and heating action Send to the vacuum furnace dynamic simulation model for execution; S05. Vacuum furnace dynamic simulation model executes heating action. New state information is then generated, and based on this, the state awareness module generates a state vector. Simultaneously, the reward receiving unit generates action rewards. ; S06. Experience replay buffer will , , and Integrating into quadruple samples Store; S07. Repeat steps S02-S06 until the number of quadruple samples stored in the experience playback buffer reaches the preset number of samples or the agent and the vacuum furnace dynamic simulation model meet the preset number of interactions, and obtain the agent after simulation training, wherein the number of interactions is the number of times steps S02-S06 are repeated. S08. Evaluate the performance of the simulated agent after training to obtain the temperature control accuracy. If the temperature control accuracy meets the preset temperature control accuracy threshold, the trained agent is obtained, i.e., the dynamic heating control model. Otherwise, return the simulated agent after training to S02 to continue simulation training.

[0012] Preferably, the method for performance evaluation of the agent after simulation training is as follows: In the dynamic simulation model of the vacuum furnace, the process target temperature curve is set. The simulated agent is connected to the dynamic simulation model of the vacuum furnace and runs according to the process target temperature curve. The real-time temperature at five different time points of the dynamic simulation model of the vacuum furnace is obtained. The real-time temperature at the five different time points is the real-time temperature at the midpoint of the heating section, the starting point of the holding section, the midpoint of the holding section, the ending point of the holding section, and the midpoint of the cooling section. Simultaneously, the target temperatures at five different time points corresponding to the process target temperature curve are read: the target temperatures at the midpoint of the heating section, the beginning point of the holding section, the midpoint of the holding section, the end point of the holding section, and the midpoint of the cooling section. Based on this, the actual temperature deviations at the five different time points are calculated, and the average of the actual temperature deviations at the five different time points is taken. That is, temperature control accuracy; Set a temperature control accuracy threshold according to process requirements, and compare the temperature control accuracy with the temperature control accuracy threshold: If the temperature control accuracy is less than or equal to the temperature control accuracy threshold, a dynamic heating control model is obtained. If the temperature control accuracy exceeds the temperature control accuracy threshold, the simulation training will be repeated.

[0013] Preferably, the process of real-time monitoring of the operating status data of the vacuum furnace to determine whether the dynamic heating control model needs to be re-simulated and trained: The process standard presets based on vacuum furnaces include overshoot thresholds. Overshoot threshold Energy consumption threshold per unit time Temperature fluctuation threshold during the insulation stage The performance threshold, while simultaneously acquiring the overshoot under vacuum furnace operation in real time. Overshoot count Energy consumption per unit time Temperature fluctuations during the insulation stage ; like and , or If the dynamic heating control model fails, it will be re-simulated and trained; otherwise, the operation of the vacuum furnace will continue to be monitored to identify control failures caused by performance degradation or changes in operating conditions of the dynamic heating control model in real time.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention improves the accuracy and stability of temperature control. Traditional fixed parameter control or empirical adjustment is difficult to adapt to the complex heat transfer characteristics of vacuum furnaces, which can easily lead to large temperature deviations and significant overshoot. This invention uses a reinforcement learning-driven intelligent agent, combined with the accurate simulation of a dynamic simulation model, to perceive the temperature and pressure status information of monitoring points in the furnace in real time. Through a strategy network, the heating action is dynamically optimized, which significantly reduces temperature deviations and reduces temperature fluctuations during the heat preservation stage, ensuring the stability of the heating process.

[0015] 2. This invention enhances adaptability and process flexibility. Traditional control strategies are poorly adaptable to equipment characteristic drift, material changes, or process curve adjustments, requiring repeated manual parameter adjustments. The reinforcement learning model of this invention has dynamic learning capabilities. The agent can automatically adapt to different material thermal properties and process target curves through interaction with the dynamic simulation model. At the same time, by monitoring the operating status data in real time, the model can be retrained to ensure that it always adapts to the actual state of the equipment during long-term use, reducing the cost of manual intervention.

[0016] 3. This invention reduces development and application costs. Traditional heating strategies rely on repeated experiments and debugging in physical vacuum furnaces, which not only consumes production time and increases material consumption, but also poses risks of high-temperature operation. This invention conducts agent training based on a dynamic simulation model of a vacuum furnace, which can cover more than a thousand working condition combinations in a virtual environment, greatly reducing the number of physical experiments, shortening the strategy development cycle, and reducing equipment consumption and safety risks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0020] Examples, such as Figure 1 As shown, a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning includes the following steps: S001. Obtain the vacuum furnace characteristic data and construct a dynamic simulation model of the vacuum furnace based on the vacuum furnace characteristic data; S002. Construct an intelligent agent, put the intelligent agent into the dynamic simulation model of the vacuum furnace for simulation training, and obtain a dynamic heating control model; S003. Integrate the dynamic heating control model into the control software of the vacuum furnace. The vacuum furnace operates by running the control software with the integrated dynamic heating control model and monitors the operating status data of the vacuum furnace in real time to determine whether the dynamic heating control model needs to be re-simulated and trained.

[0021] Furthermore, the working principle of the present invention will be illustrated below through embodiments: Taking the high-temperature annealing treatment of 304 stainless steel workpieces in a certain type of industrial vacuum furnace as an example, the method of the present invention is used to achieve dynamic optimization of the heating strategy.

[0022] The characteristic data of the vacuum furnace were acquired and a dynamic simulation model of the vacuum furnace was constructed. For this vacuum furnace, the collected characteristic data included heating system parameters, furnace chamber structure parameters, vacuum system parameters, control system parameters, heat transfer parameters, and material thermal properties. The heating system parameters included that the heating element was a graphite heater with a rated power of 60kW, consisting of four groups (distributed on both sides and the top of the furnace chamber), and a heating response time of less than 5 seconds. The furnace chamber structure parameters included furnace chamber dimensions of 1200mm × 800mm × 600mm, and an insulation layer of alumina fiber with a thickness of 1... 50mm; Vacuum system parameters include a rotary vane vacuum pump, ultimate vacuum degree not greater than 1Pa, pumping speed of 20L / s, furnace pressure adjustment range of 1-100Pa; Control system parameters include PLC controller response delay not greater than 100ms and temperature sensor is type K thermocouple; Heat transfer parameters include furnace internal radiation coefficient of 0.85 and solid thermal conductivity of 0.15W / (m・K); Material thermal properties include specific heat capacity of 304 stainless steel of 0.5kJ / (kg・K), melting point of 1400℃ and latent heat of phase change of 272kJ / kg.

[0023] Based on the characteristic data of the vacuum furnace, a dynamic simulation model of the vacuum furnace was constructed using the COMSOL Multiphysics simulation platform. A three-dimensional geometric model was established based on the furnace structure parameters to restore the actual layout of the heater and thermocouples. Heat transfer parameters were imported to define the radiation and conduction heat transfer coefficients in the furnace. Combined with the vacuum system parameters, the dynamic change equations of furnace pressure and gas composition were set. At the same time, the heating system and control system parameters were integrated to simulate the dynamic response of heating power output and temperature sensors. The thermal characteristic parameters of 304 stainless steel were also embedded, so that the dynamic simulation model of the vacuum furnace can accurately simulate the temperature change during the workpiece heating process and output status information in real time.

[0024] Construct an intelligent agent, configure its structure, and have the state perception module acquire the temperature of five monitoring points in the simulation model in real time. (located in the center and four corners of the furnace, respectively) real-time heating power Furnace pressure Gas flow rate After normalization, it is converted into an 8-dimensional state vector. The policy network employs a multilayer perceptron (MLP), with an 8-dimensional input layer, 128 and 64 hidden layers, and a 5-dimensional output layer corresponding to the probability distribution of five discrete heating power adjustment actions (-5kW, -3kW, 0kW, +3kW, +5kW). The experience replay buffer has a capacity of 100,000 records, storing quadruples of samples. .

[0025] The constructed agent is placed into a vacuum furnace dynamic simulation model for simulation training. The parameters of the policy network and value network are initialized, and the initial state of the vacuum furnace dynamic simulation model is set. The agent then interacts with the vacuum furnace dynamic simulation model, and the state perception module generates a state vector. The policy network outputs the action probability distribution, and the action execution unit selects the heating action. After the vacuum furnace dynamic simulation model is executed, the next state vector is generated. The reward receiving unit calculates based on the reward function. The experience replay buffer will , , and Integrating into quadruple samples The data is stored and repeatedly interacted with in the buffer until 100,000 quadruple samples are obtained. Training is then stopped. The target temperature curve is set as "room temperature → 5℃ / min heating to 1050℃ → heat preservation for 2h → 2℃ / min cooling to 200℃". Five time points are selected: t1=500℃, t2=1050℃, t3=heat preservation for 1h (1050℃), t4=heat preservation end (1050℃), and t5=500℃. The temperature deviation is calculated, and the average value is 1.8℃, which meets the preset threshold (≤±2℃). The dynamic heating control model is thus obtained.

[0026] The dynamic heating control model is compiled into a C++ dynamic link library and integrated into the PLC control software of the vacuum furnace. Real-time data interaction between the model and the vacuum furnace sensors and actuators is achieved through the OPC protocol. During vacuum furnace operation, the overshoot is collected in real time. (Maximum difference between actual temperature and 1050℃), number of overshoots (Cumulative number of overshoots), energy consumption per unit time Temperature fluctuations during the insulation stage (The difference between the highest and lowest temperatures during the heat preservation period), and based on the process standard presets of the vacuum furnace, including the overshoot threshold. The overshoot threshold is 5℃. For 2 times, the energy consumption threshold per unit time The threshold for temperature fluctuation during the heat preservation stage is 45 kW·h / h. The performance threshold is 3℃. If it is detected... 6℃ and For 3 times (or For 48 kW·h / h, or If the temperature is 4℃, then a re-simulation training will be triggered; otherwise, the process will continue.

[0027] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning, characterized in that, Includes the following steps: S1. Obtain vacuum furnace characteristic data and construct a dynamic simulation model of the vacuum furnace based on the vacuum furnace characteristic data; S2. Construct an agent that includes a state perception module, a policy network, a value network, an experience playback buffer, a parameter update unit, an action execution unit, and a reward receiving unit. Place the agent into a dynamic simulation model of a vacuum furnace for simulation training to obtain a dynamic heating control model. The state perception module is used to acquire state information in real time from the dynamic simulation model of the vacuum furnace and convert the state information into a state vector. ; The policy network is used to express state vectors. As input, it is processed through a multi-layer neural network, and the output is a state vector. The corresponding probability distribution of heating actions; The value network is used to receive state vectors. Output state vector Corresponding state value estimate ; The experience replay buffer is used to receive the state vector. and Heating action and action rewards and will receive the state vector and Heating action and action rewards Integrating into quadruple samples Store, where The vacuum furnace dynamic simulation model executes the heating action. The next state vector; The parameter update unit is used to obtain the quadruplet samples stored in the experience replay buffer. and value network output Based on this, the loss functions of the policy network and the value network are calculated, and the parameters of the policy network and the value network are updated using the gradient descent algorithm according to the loss functions of the policy network and the value network. The action execution unit is used to select a heating action based on the action probability distribution. and heating action Send to the vacuum furnace dynamic simulation model for execution; The reward receiving unit is used to obtain the dynamic simulation model of the vacuum furnace when performing heating actions. Post-action reward ; S3. Integrate the dynamic heating control model into the control software of the vacuum furnace. The vacuum furnace operates by running the control software with the integrated dynamic heating control model and monitors the operating status data of the vacuum furnace in real time to determine whether the dynamic heating control model needs to be re-simulated and trained.

2. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The vacuum furnace characteristic data includes heating system parameters, furnace chamber structure parameters, vacuum system parameters, control system parameters, heat transfer parameters, and material thermal properties.

3. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 2, characterized in that, The method for constructing a dynamic simulation model of a vacuum furnace based on vacuum furnace characteristic data: Based on the characteristic data of the vacuum furnace, a dynamic simulation model of the vacuum furnace is constructed using computer simulation technology.

4. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, In the reward receiving unit, the dynamic simulation model of the vacuum furnace is acquired during the execution of the heating action. Post-action reward The process: The dynamic simulation model of the vacuum furnace executes the heating action. Then, new state information is generated. The reward receiving unit receives the new state information and uses it to calculate the action reward using the reward function formula. The new status information includes heating power. Furnace pressure Gas flow rate and monitoring point temperature .

5. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 4, characterized in that, The method of placing the intelligent agent into a dynamic simulation model of a vacuum furnace for simulation training: S01. Initialize the parameters of the agent's policy network and value network, and initialize the vacuum furnace dynamic simulation model, and set the initial state information of the vacuum furnace dynamic simulation model. S02. The state awareness module acquires initial state information in real time from the dynamic simulation model of the vacuum furnace and outputs a state vector. ; S03. The policy network uses state vectors. Input is the state vector, output is the state vector. The corresponding action probability distribution, and the value network receives the state vector. Output state vector Corresponding state value estimate ; S04. The action execution unit selects the heating action based on the action probability distribution. and heating action Send to the vacuum furnace dynamic simulation model for execution; S05. Vacuum furnace dynamic simulation model executes heating action. New state information is then generated, and based on this, the state awareness module generates a state vector. Simultaneously, the reward receiving unit generates action rewards. ; S06. Experience replay buffer will , , and Integrating into quadruple samples Store; S07. Repeat steps S02-S06 until the training conditions are met to obtain the simulated trained agent. S08. Evaluate the performance of the simulated agent after training to obtain the temperature control accuracy. If the temperature control accuracy meets the preset temperature control accuracy threshold, the trained agent is obtained, i.e., the dynamic heating control model. Otherwise, return the simulated agent after training to S02 to continue simulation training.

6. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 5, characterized in that, The training conditions are that the number of quadruple samples stored in the experience playback buffer reaches a preset number of samples or the agent and the vacuum furnace dynamic simulation model meet a preset number of interactions. The number of interactions is the number of times steps S02-S06 are repeated.

7. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 6, characterized in that, The method for performance evaluation of the simulated trained agent: In the dynamic simulation model of the vacuum furnace, a process target temperature curve is set. The simulated agent is run according to the process target temperature curve and the real-time temperature at five different time points of the dynamic simulation model of the vacuum furnace is obtained. At the same time, the target temperature at the corresponding five different time points in the process target temperature curve is read. Based on this, the actual temperature deviation at the five different time points is calculated, and the average value of the actual temperature deviation at the five different time points is taken, which is the temperature control accuracy.

8. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The process of real-time monitoring of the operating status data of the vacuum furnace and determining whether to re-simulate and train the dynamic heating control model: The process standard presets based on vacuum furnaces include overshoot thresholds. Overshoot threshold Energy consumption threshold per unit time Temperature fluctuation threshold during the insulation stage The performance threshold, while simultaneously acquiring the overshoot under vacuum furnace operation in real time. Overshoot count Energy consumption per unit time Temperature fluctuations during the insulation stage ; like and , or If the simulation fails, the dynamic heating control model will be retrained; otherwise, the operation of the vacuum furnace will continue to be monitored.

Images