A vacuum furnace heating strategy dynamic adjustment method based on reinforcement learning

Abstract

This invention discloses a dynamic adjustment method for heating strategy of a vacuum furnace based on reinforcement learning, relating to the field of industrial process control technology. The method includes: acquiring a smoothed vacuum degree sequence and a multi-temperature zone temperature sequence of the vacuum furnace; solving these sequences using a modal gating function to obtain convective heat transfer weights and radiative heat transfer weights; constructing heat flow features based on these weights to obtain a dynamic physical state vector; calculating a dynamic causal edge weight matrix using the dynamic physical state vector; and performing multi-round node feature aggregation through an attention-gated graph convolution mechanism to obtain a graph embedding feature vector; inputting the graph embedding feature vector, the dynamic causal edge weight matrix, and a historical action window into a causal credit evaluation network; and constraining message passing through a graph attention mechanism to generate temporal credit allocation coefficients. This invention achieves real-time quantization of the proportion of heat transfer modes and high-fidelity physical reconstruction of the state space under continuous gas extraction and release conditions in a vacuum furnace.

Description

Technical Field

[0001] This invention relates to the field of industrial process control technology, and in particular to a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning. Background Technology

[0002] In the field of vacuum heat treatment, the dynamic adjustment of heating strategies directly determines the evolution of workpiece microstructure and process consistency. While traditional PID control and model predictive control are the mainstream in industry, the former heavily relies on expert experience for tuning and has weak disturbance rejection capabilities, making it difficult to adapt to fluctuations in furnace loading and aging of insulation panels; the latter, although capable of handling multiple constraints, is prone to model mismatch under the strong nonlinearity of vacuum radiation heat transfer and the time-varying coupling of the perspective factor. In recent years, reinforcement learning, with its model-free online adaptive characteristics, has been introduced into the field of complex thermal control, initially verifying the feasibility of data-driven strategies in nonlinear temperature tracking tasks.

[0003] However, existing reinforcement learning schemes still have significant limitations in vacuum furnace applications: First, conventional methods often simplify the vacuum level to a static scalar, failing to characterize the dynamic migration of convection-radiation heat transfer modes caused by the migration of residual gas during the evacuation and degassing stages, resulting in distortion of the state space representation and a sharp drop in the accuracy of variable operating condition control; Second, the large thermal inertia and multi-region thermal coupling characteristics cause the traditional exponential discount time-series difference mechanism to fail, making it impossible to accurately separate the true causal contribution of historical power commands to the current temperature field transition, resulting in policy gradient updates being affected by pseudo-correlation noise and convergence oscillations. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning to solve the problem of strategy convergence oscillation caused by distortion of heat transfer mode representation and failure of causal attribution due to long thermal hysteresis under variable vacuum conditions.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning. The method includes: acquiring a smoothed vacuum degree sequence and a multi-temperature zone temperature sequence of the vacuum furnace; solving these sequences using a modal gating function to obtain convective heat transfer weights and radiative heat transfer weights; constructing heat flow features based on these weights to obtain a dynamic physical state vector; calculating a dynamic causal edge weight matrix using the dynamic physical state vector; and performing multi-round node feature aggregation using an attention-gated graph convolution mechanism to obtain a graph embedding feature vector; inputting the graph embedding feature vector, the dynamic causal edge weight matrix, and the historical action window into a causal credit evaluation network, and then using the graph... Attention mechanisms constrain message passing, generating temporal credit allocation coefficients. Based on these coefficients, historical control cycles are weighted and integrated to obtain a modified advantage function. Backpropagation of this modified advantage function yields the policy network gradient and value network gradient. The policy network gradient and value network gradient are used to optimize the decision output distribution, generating a multi-temperature zone original heating strategy. A safety projection is performed through a control barrier function to generate a safe power control command. The safe power control command is executed, and actual temperature data for the next cycle is collected. Predictive residuals are calculated, and incremental fine-tuning of the causal credit assessment network is performed when residuals exceed limits.

[0007] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for obtaining the convective heat transfer weights and the radiative heat transfer weights are as follows: The raw vacuum level data and raw temperature data of each heating zone of the vacuum furnace are collected, and the raw vacuum level data are smoothed and time-series aligned to obtain a smoothed vacuum level sequence and a multi-temperature zone temperature sequence. Based on the smooth vacuum degree sequence, the multi-temperature zone temperature sequence and the preset initial heat transfer parameters, the intensity of convective candidate heat flux and the intensity of radiative candidate heat flux are calculated. Based on the convective candidate heat flux intensity and the radiative candidate heat flux intensity, normalization mapping is performed through a modal gating function to obtain the convective heat transfer weight and the radiative heat transfer weight.

[0008] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for obtaining the dynamic physical state vector are as follows: By linearly combining the heat transfer states of each heating zone with the convective heat transfer weight and the radiative heat transfer weight, the heat flow characteristics corresponding to each heating zone are constructed. By assembling the heat flow characteristics, smoothed vacuum degree sequence, and multi-temperature zone temperature sequence, a dynamic physical state vector is obtained.

[0009] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace, the specific steps for calculating the dynamic causal edge weight matrix using the dynamic physical state vector are as follows: Extract multi-dimensional feature vectors of each node from the dynamic physical state vector and the temperature gradient difference between adjacent regions; Based on the multidimensional feature vectors of each node and the temperature gradient difference between adjacent regions, and combined with the convective heat transfer weight and the radiative heat transfer weight, the dynamic causal edge weight matrix is ​​calculated through the modal causal edge weight function.

[0010] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for obtaining the graph embedding feature vector are as follows: Based on the dynamic causal edge weight matrix and the multidimensional feature sub-vectors of each node, multiple rounds of node feature aggregation are performed through the attention-gated graph convolution mechanism to obtain the iterative node embedding set. Global pooling and dual-channel fusion, along with row normalization probability mapping, are applied to the node embedding set to obtain the graph embedding feature vector.

[0011] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for generating the temporal credit allocation coefficients are as follows: The graph embedding feature vector, dynamic causal edge weight matrix and historical action window are input into the causal credit assessment network for time-series correlation encoding to generate a graph conditional action event sequence. The key historical action set is then extracted from the graph conditional action event sequence to obtain the historical backtracking index. A temporal credit graph of actions is constructed using historical backtracking indexes, and a graph attention mechanism is used to constrain the message passing of nodes in the temporal credit graph of actions using a dynamic causal edge weight matrix, thereby obtaining a hidden representation of credit propagation. By performing prefix reward decomposition on the hidden representation of credit propagation, the marginal causal contribution is obtained; The marginal causal contribution, historical backtracking index, and dynamic causal weight matrix are normalized and calculated to generate time-series credit allocation coefficients.

[0012] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for obtaining the policy network gradient and the value network gradient are as follows: By aligning the time-series credit allocation coefficients with the time-series difference errors corresponding to the historical control periods, a credit-weighted time-series difference sequence is obtained. The modified advantage function is obtained by weighted integral fusion of the credit-weighted time-series difference sequences. The modified advantage function is used to constrain the policy distribution corresponding to the historical time series difference error and the historical control action, and the value network loss function and policy network loss function are constructed. The value network gradient is obtained by backpropagating through the value network loss function, and the policy network gradient is obtained by backpropagating through the policy network loss function.

[0013] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for generating the original heating strategy for multiple temperature zones are as follows: A joint state representation is constructed based on dynamic physical state vectors, graph embedding feature vectors, and dynamic causal edge weight matrices. The distribution parameters are then mapped by combining the gradients of the policy network and the value network to obtain the initial parameters of the decision output distribution. Based on the initial parameters of the decision output distribution, the power allocation probability corresponding to each heating zone is optimized and inferred by combining the gradient of the policy network and the gradient of the value network, so as to obtain the target heating allocation vector of the multi-temperature zone. By combining the target heating allocation vector of the multi-temperature zone with the actual power distribution of the previous control cycle, smooth migration decoding is performed to obtain the original heating strategy of the multi-temperature zone.

[0014] As a preferred embodiment of the reinforcement learning-based method for dynamically adjusting the heating strategy of a vacuum furnace according to the present invention, the specific steps for generating safe power control commands are as follows: Based on the original multi-temperature zone heating strategy, the current temperature state of the multi-temperature zone, and the dynamic causal edge weight matrix, the temperature response prediction for the next control cycle is constructed. Based on the temperature response prediction, a set of control barrier function constraints is constructed by combining the upper limit of temperature in each heating zone, the upper limit of temperature difference between adjacent temperature zones, and the power change rate constraint. The original heating strategy of the multi-temperature zone is then subjected to safety projection processing using the set of control barrier function constraints to obtain the candidate vector of safe power for the multi-temperature zone. Based on the candidate vector of safe power for multiple temperature zones, boundary correction is performed by combining the upper and lower limits of power for each heating zone and the power smoothing constraints of adjacent control cycles to obtain the safe power control command.

[0015] As a preferred embodiment of the reinforcement learning-based dynamic adjustment method for vacuum furnace heating strategy described in this invention, the steps of executing a safe power control command and collecting actual temperature data for the next cycle, calculating the prediction residual, and incrementally fine-tuning the causal credit evaluation network when the residual exceeds the limit are as follows: Execute the safety power control command and collect the actual temperature data of each heating zone in the next control cycle; The predicted residuals for each heating zone are calculated by comparing the actual temperature data with the predicted temperature response of the corresponding control cycle. The predicted residuals and residual thresholds are used to determine if they exceed the limit. When the predicted residuals exceed the limit, an incremental fine-tuning sample corresponding to the current control period is constructed, and the local parameters of the causal credit assessment network are incrementally fine-tuned based on the incremental fine-tuning sample.

[0016] The beneficial effects of this invention are as follows: By solving the convection and radiation heat transfer weights using modal gating functions and constructing a dynamic physical state vector, real-time quantization of the proportion of heat transfer modes and high-fidelity physical reconstruction of the state space are achieved under continuous gas extraction and venting conditions in a vacuum furnace. This achieves the effects of maintaining the fidelity of state characterization and improving the accuracy of temperature field tracking under variable vacuum conditions, and effectively suppressing the oscillation of control commands and the frequent triggering of safety constraints. By generating temporal credit allocation coefficients through dynamic causal edge weight calculation and attention graph convolution constraints, the true causal separation of historical power actions on the current temperature field transition in a multi-region strongly coupled and large thermal hysteresis system is realized. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning.

[0019] Figure 2 A flowchart for heat transfer mode sensing and dynamic physical state construction.

[0020] Figure 3 A flowchart for strategy optimization and secure projection control.

[0021] Figure 4 The flowchart is for closed-loop execution and incremental fine-tuning.

[0022] Figure 5 This is a heatmap of the dynamic causal edge weight matrix.

[0023] Figure 6 This is a distribution diagram of time-series credit allocation coefficients. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Reference Figures 1-6 This is one embodiment of the present invention, which provides a method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning, including the following steps: S1. Collect the smooth vacuum degree sequence and multi-temperature zone temperature sequence of the vacuum furnace, solve them through the modal gating function to obtain the convective heat transfer weight and the radiative heat transfer weight, and construct the heat flow characteristics based on the convective heat transfer weight and the radiative heat transfer weight to obtain the dynamic physical state vector.

[0028] The raw vacuum level data of the vacuum furnace and the raw temperature data of each heating zone are collected. The raw vacuum level data is then smoothed and time-series aligned to obtain a smoothed vacuum level sequence and a multi-temperature zone temperature sequence.

[0029] The specific process includes continuously collecting raw vacuum data and raw temperature data of each heating zone at a fixed sampling frequency using vacuum sensors and temperature sensors installed on the vacuum furnace body. For the collected raw vacuum data, high-frequency noise is eliminated by sliding window averaging or low-pass filtering, while retaining low-frequency signals that reflect the true pressure inside the furnace. The raw temperature data of each heating zone are aligned according to the timestamp of the control cycle, and abnormal outliers caused by sensor failure or electromagnetic interference are removed to obtain a smooth vacuum sequence and a multi-temperature zone temperature sequence.

[0030] Based on a smoothed vacuum degree sequence, a multi-temperature zone temperature sequence, and preset initial heat transfer parameters, the convective candidate heat flux intensity and the radiative candidate heat flux intensity are calculated, and the expressions are as follows: ; ; in, Indicates the first Convection candidate heat flux intensity in the heating zone Indicates the index of the heating zone. This represents the convective heat transfer coefficient at atmospheric pressure. Represents the smoothed vacuum degree sequence value. This indicates the reference atmospheric pressure value. Indicates the pressure-convection coupling index. Indicates the first Smoothed temperature sequence values ​​of the heating zone Indicates the equivalent temperature of the residual gas inside the furnace. Indicates the first The intensity of radiative candidate heat flux in the heating zone. This indicates the initial emissivity calibration value of the furnace inner wall and heating elements. Represents the Stefan-Boltzmann radiation constant. This represents the isothermal boundary value of the water-cooled furnace wall. Indicates the enhancement factor of scattered radiation in the medium vacuum region. This indicates the critical pressure for switching heat transfer modes.

[0031] It should be noted that, The step power response of the vacuum furnace was measured under standard atmospheric pressure, and steady-state temperature rise data was collected and calibrated by inversion based on Newton's law of cooling. It was obtained by power-law regression fitting of convective heat transfer experimental data under different vacuum gradients, and calibrated according to the scaling relationship between the Nusselt number and the Knudsen number in the theory of rarefied gas dynamics. It was obtained by conducting a special calibration of radiative heat transfer in the medium vacuum pressure range, measuring the scattering attenuation ratio of residual gas molecules to thermal radiation flux, and performing nonlinear least squares fitting calibration in combination with the radiative transfer equation in the kinetic theory of gases. It was obtained by power-law regression fitting of convective heat transfer experimental data under different vacuum gradients, and calibrated according to the scaling relationship between the Nusselt number and the Knudsen number in the theory of rarefied gas dynamics. It is based on the fourth variance of the initial emissivity calibration value of the furnace inner wall, the radiation constant, the smooth temperature sequence and the isothermal value of the water-cooled furnace wall, and combined with the scattered radiation enhancement coefficient of the medium vacuum region and the real-time vacuum degree correction term. The radiative heat flux at a known temperature was measured and calibrated by inversion based on the Stefan-Boltzmann law through a blackbody radiation control experiment conducted under standard atmospheric pressure and high vacuum conditions.

[0032] The specific process includes: based on the smoothed vacuum degree sequence, the multi-temperature zone temperature sequence, and the preset initial heat transfer parameters, using physical quantities such as the atmospheric pressure reference convective heat transfer coefficient, the pressure-convective coupling index, the reference atmospheric pressure reference value, and the equivalent temperature of the residual gas in the furnace, combined with the smoothed vacuum degree sequence value and the smoothed temperature sequence value of the heating zone, the convective candidate heat flux intensity of the heating zone is obtained through power function operation and difference calculation; using physical quantities such as the initial emissivity calibration value of the furnace inner wall and heating elements, the Stefan-Boltzmann radiation constant, the isothermal boundary value of the water-cooled furnace wall, the scattering radiation enhancement coefficient of the medium vacuum zone, and the critical pressure for heat transfer mode switching, combined with the smoothed vacuum degree sequence value and the smoothed temperature sequence value of the heating zone, the radiation candidate heat flux intensity of the heating zone is obtained through fourth-order difference value operation and linear combination operation.

[0033] It should be noted that the initial heat transfer parameters refer to the pre-set coefficients or variables used to describe the heat transfer efficiency, which are preset based on the statistical average of historical operating conditions.

[0034] Based on the convective candidate heat flux intensity and the radiative candidate heat flux intensity, normalization mapping is performed through a modal gating function to obtain the convective heat transfer weight and the radiative heat transfer weight.

[0035] The specific process includes: based on the convective candidate heat flux intensity and the radiative candidate heat flux intensity, normalization mapping is performed through a modal gating function; the convective candidate heat flux intensity and the radiative candidate heat flux intensity are placed under a unified calculation scale for intensity mapping and proportional allocation; the relative difference between the convective candidate heat flux intensity and the radiative candidate heat flux intensity is continuously quantified; and the convective candidate heat flux intensity and the radiative candidate heat flux intensity form a mutually constrained weight relationship in the same mapping process; so that when the convective candidate heat flux intensity increases, the convective heat transfer weight increases synchronously; when the radiative candidate heat flux intensity increases, the radiative heat transfer weight increases synchronously; and the convective heat transfer weight and the radiative heat transfer weight maintain a corresponding state of total uniformity, thus obtaining convective heat transfer weight and radiative heat transfer weight consistent with the current heat transfer conditions.

[0036] By linearly combining the heat transfer states of each heating zone with the convective heat transfer weight and the radiative heat transfer weight, the heat flow characteristics corresponding to each heating zone are constructed.

[0037] The specific process includes linearly combining the heat transfer states of each heating zone using convective heat transfer weights and radiative heat transfer weights. The heat transfer states of each heating zone are simultaneously subject to the combined effects of convective and radiative heat transfer weights, so that the contributions of each heating zone's heat transfer state in the convective heat transfer direction and the contributions in the radiative heat transfer direction are uniformly quantified. The combination ratio of the heat transfer states of each heating zone is adjusted according to the distribution difference of convective and radiative heat transfer weights, forming a comprehensive characterization that can reflect the current heat transfer strength relationship and heat transfer composition differences of each heating zone, thereby constructing the heat flow characteristics corresponding to each heating zone.

[0038] It should be noted that the weight ratio is preset based on the proportion of convective candidate heat flux intensity and radiative candidate heat flux intensity in the total heat transfer process. The weight allocation ratio of convection and radiation is determined by heat transfer theory. Heat flux characteristics refer to the characteristic information used to characterize the heat transfer state of each heating zone, which can reflect the strength of heat transfer in each heating zone and the combined effect of convective and radiative heat transfer.

[0039] By assembling the heat flow characteristics, smoothed vacuum degree sequence, and multi-temperature zone temperature sequence, a dynamic physical state vector is obtained.

[0040] The specific process includes: heat flow characteristics characterizing the comprehensive heat flow state of each heating zone; smoothed vacuum degree sequence characterizing the changing trend of the pressure environment inside the furnace; and multi-temperature zone temperature sequence characterizing the temperature distribution state of each heating zone. The heat flow characteristics, smoothed vacuum degree sequence, and multi-temperature zone temperature sequence are arranged in a preset order to form a dynamic physical state vector containing three types of physical quantities: heat flow, pressure, and temperature. The dynamic physical state vector is used to uniformly characterize the multi-dimensional features of the thermal state inside the vacuum furnace.

[0041] It should be noted that the preset sequence is preset according to a fixed arrangement rule of heat flow characteristics, smooth vacuum degree sequence, and multi-temperature zone temperature sequence.

[0042] S2. Calculate the dynamic causal edge weight matrix using the dynamic physical state vector, and perform multi-round node feature aggregation through the attention-gated graph convolution mechanism to obtain the graph embedding feature vector.

[0043] Extract multi-dimensional feature vectors of each node from the dynamic physical state vector and the temperature gradient differences between adjacent regions.

[0044] The specific process includes separating the multi-dimensional feature sub-vectors corresponding to each node from the dynamic physical state vector through slicing operations, obtaining the temperature gradient difference between adjacent heating zones through difference operations, and the multi-dimensional feature sub-vectors of each node containing the heat flow, pressure, and temperature information of the corresponding node. The temperature gradient difference between adjacent zones reflects the temperature change rate between adjacent heating zones. The extraction process decomposes the dynamic physical state vector into two types of features: the multi-dimensional feature sub-vectors of each node and the temperature gradient difference between adjacent zones.

[0045] Based on the multidimensional feature vectors of each node and the temperature gradient differences between adjacent regions, and combining the convective heat transfer weights and radiative heat transfer weights, the dynamic causal edge weight matrix is ​​calculated using the modal causal edge weight function, expressed as follows: ; in, Indicates the current control cycle Lower source heating zone To the target heating area The dynamic causal edge weight matrix, Indicates the index of the lower source heating zone. Indicates the target heating zone index. This indicates the time index of the current control cycle. Indicates the radiative heat transfer weight. This represents the natural exponential function. Indicates the lower source heating zone The multidimensional feature vectors, Indicates the target end heating area The multidimensional feature vectors, This represents the radiation characteristic attenuation scale parameter. Indicates the convective heat transfer weight. Indicates the lower source heating zone heating zone at the target end The temperature gradient difference between adjacent regions This represents the baseline value for sensitivity to convection differences. This indicates the total number of heating zones in the vacuum furnace. Represents any node in the heating zone. Represents heating zone nodes The multidimensional feature vectors, This represents a numerically stable regularized infinitesimal element.

[0046] It should be noted that, It is obtained by normalizing and mapping the convective candidate heat flux intensity and the radiative candidate heat flux intensity through a modal gate function; It is obtained by normalizing and mapping the convective candidate heat flux intensity and the radiative candidate heat flux intensity through a modal gate function; It was obtained through nonlinear regression fitting of temperature difference and power response delay data between adjacent zones under historical operating conditions.

[0047] The specific process includes: using the natural exponential function to exponentially decay the squared Euclidean distance of the multidimensional feature vectors; combining the radiative heat transfer weights to form a radiative heat transfer contribution term; performing inverse scaling on the temperature gradient difference between adjacent regions; combining the convective heat transfer weights to form a convective heat transfer contribution term; summing the radiative heat transfer contribution term and the convective heat transfer contribution term; normalizing the sum of the radiative heat transfer contribution term and the convective heat transfer contribution term for all heating zone nodes; and superimposing numerically stable regularized infinitesimal elements to obtain the dynamic causal edge weight matrix from the source heating zone to the target heating zone under the current control cycle.

[0048] It should be noted that the modal causal edge weight function is used to calculate the causal correlation strength between multiple heating zones based on the convective heat transfer weight, the radiative heat transfer weight, and the state differences between each heating zone. The modal causal edge weight function plays the role of converting heat transfer mode information into edge weight information, so that the coupling relationship between multiple heating zones can change dynamically with the current heat transfer state. The significance of the modal causal edge weight function is to improve the ability of the causal graph to represent the real heat transfer process, and to provide a more accurate correlation basis for subsequent node feature aggregation and graph embedding feature extraction.

[0049] like Figure 5 The dynamic causal edge weight matrix heatmap displays the distribution of causal influence intensity between heating zones within the current control cycle. The horizontal axis represents the target heating zone, and the vertical axis represents the source heating zone. Color depth indicates the magnitude of the dynamic causal edge weight between corresponding heating zones; darker colors indicate a stronger dynamic influence of the source heating zone on the target heating zone. This graph is calculated using a modal causal edge weight function, based on multi-dimensional feature vectors extracted from each node in the dynamic physical state vector and the temperature gradient differences between adjacent zones, combined with convective and radiative heat transfer weights. Therefore, it can intuitively reflect the coupling strength distribution, key influence channels, and propagation direction of local disturbances among the multiple temperature zones of the vacuum furnace under the current operating conditions. This heatmap demonstrates that the present invention does not employ fixed temperature zone relationships, but rather dynamically characterizes the real-time influence relationships between temperature zones as the smoothed vacuum degree sequence, temperature distribution state, and heat transfer mode proportions change. This provides a more realistic structural basis for subsequent graph embedding feature vector generation and time-series credit allocation coefficient calculation.

[0050] Based on the dynamic causal edge weight matrix and the multidimensional feature sub-vectors of each node, multiple rounds of node feature aggregation are performed through the attention-gated graph convolution mechanism to obtain the iterative node embedding set.

[0051] The specific process includes: concatenating the multidimensional feature vectors of each node with the dynamic causal edge weight matrix after linear transformation; generating attention weight coefficients through an activation function; using the attention weight coefficients to perform a weighted summation of the multidimensional feature vectors of each node to obtain the aggregated node feature representation; performing a residual connection between the aggregated node feature representation and the original multidimensional feature vectors of each node; obtaining the updated node embedding through a nonlinear transformation; repeating multiple rounds of node feature aggregation and embedding update to obtain the iterative node embedding set.

[0052] It should be noted that the attention-gated graph convolution mechanism obtains the attention weight coefficients between nodes through a dynamic causal edge weight matrix, uses the attention weight coefficients to perform weighted aggregation of the multi-dimensional feature sub-vectors of each node, and updates the node embeddings by combining residual connections and nonlinear transformations; in multiple iterations, the node feature representation is gradually optimized to obtain the iterative node embedding set.

[0053] Global pooling and dual-channel fusion, along with row normalization probability mapping, are applied to the node embedding set to obtain the graph embedding feature vector.

[0054] The specific process includes extracting the global mean feature of the node embedding set through average pooling operation, extracting the global extreme feature of the node embedding set through max pooling operation, concatenating and fusing the global mean feature and the global extreme feature to obtain the dual-channel fused feature, performing row normalization on the dual-channel fused feature, mapping each row feature value to a probability distribution form, and obtaining the graph embedding feature vector.

[0055] S3. Input the graph embedding feature vector, dynamic causal edge weight matrix and historical action window into the causal credit evaluation network, and generate time-series credit allocation coefficients by constraining message passing through the graph attention mechanism.

[0056] The graph embedding feature vector, dynamic causal edge weight matrix, and historical action window are input into the causal credit assessment network for temporal correlation encoding to generate a graph conditional action event sequence. The key historical action set is then extracted from the graph conditional action event sequence to obtain the historical backtracking index.

[0057] The specific process includes: using a causal credit assessment network to transform the graph embedding feature vector through a fully connected layer to obtain action event features; performing matrix multiplication on the action event features and the dynamic causal edge weight matrix to obtain spatiotemporal correlation features; concatenating the spatiotemporal correlation features with the action sequence in the historical action window to obtain a joint feature representation; using a long short-term memory mechanism to perform sequence modeling on the joint feature representation to generate a graph-conditional action event sequence; selecting action events with confidence levels higher than the dynamic selection threshold from the graph-conditional action event sequence to form a set of key historical actions; and extracting the timestamp index corresponding to each action event in the set of key historical actions to obtain a historical backtracking index.

[0058] It should be noted that the dynamic screening threshold is adaptively calculated based on the confidence distribution statistics of the historical action window, and is usually determined by the mean plus or minus the standard deviation or percentile method within the sliding window. The exemplary value range is between 0.5 and 0.95. The long short-term memory mechanism works in concert with the memory cell state and the gating unit, and uses the forget gate, input gate and output gate to regulate the flow of information, effectively solving the gradient vanishing problem in long sequence processing. It can capture long-distance dependencies in the data, realize long-term storage and selective updating of historical information, and is suitable for complex time series modeling tasks. The key historical action set refers to the part of the historical control actions in the historical action window that have a strong impact on the changes in the current control cycle state and the control effect. The key historical action set includes the control actions at each historical moment that have a high causal correlation with the current control cycle and can make a major contribution to the current temperature response, prediction residual or credit allocation results.

[0059] Furthermore, during the training process of the causal credit assessment network, dynamic physical state vectors and historical action windows are received to perform forward inference. A graph attention mechanism is used to constrain node message passing and generate a credit propagation hidden representation. Prefix reward decomposition is performed on the credit propagation hidden representation to extract marginal causal contribution, and this is normalized using the real-time edge weight matrix to output temporal credit allocation coefficients. The temporal credit allocation coefficients and temporal difference errors are combined to obtain the mean square error, constructing an optimization objective function. The gradient descent algorithm is used to update the network weight parameters along the gradient direction. Incremental fine-tuning samples are collected to perform local parameter incremental fine-tuning on the causal credit assessment network until the deviation between the output temporal credit allocation coefficients and the actual response converges, completing the training.

[0060] A temporal credit graph of actions is constructed using historical backtracking indexes. By combining a dynamic causal edge weight matrix with a graph attention mechanism, the message passing of nodes in the temporal credit graph is constrained, resulting in a hidden representation of credit propagation.

[0061] The specific process includes arranging the timestamps in the historical backtracking index in chronological order, constructing directed edges to connect action nodes at different time points based on the chronological order of the timestamps to form an action time-series credit graph, constraining the message passing of nodes in the action time-series credit graph through a graph attention mechanism using a dynamic causal edge weight matrix, obtaining the attention weights between adjacent nodes in the action time-series credit graph using the graph attention mechanism, weighting and fusing the attention weights with the causal strength in the dynamic causal edge weight matrix to obtain the constrained message passing weights, aggregating and updating the node features in the action time-series credit graph based on the constrained message passing weights, and obtaining the credit propagation hidden representation through the message passing and feature aggregation operations of the graph attention mechanism.

[0062] It should be noted that the graph attention mechanism stacks multiple graph attention layers, enabling nodes to aggregate information from a wider neighborhood layer by layer, thereby capturing graph structure features from local to global. Each layer independently obtains the attention weights of neighboring nodes and updates features, achieving deep abstraction and expression of node representation through multi-layer propagation.

[0063] By performing prefix reward decomposition on the hidden representation of credit propagation, the marginal causal contribution is obtained.

[0064] The specific process includes: performing prefix reward decomposition on the hidden representation of credit propagation; obtaining the hidden representation of credit propagation through counterfactual analysis in the causal reasoning framework; assessing the incremental effects between the baseline and observed states using counterfactual analysis; evaluating the independent contribution of each feature dimension in the hidden representation of credit propagation to the final reward; quantifying the marginal causal contribution of each feature dimension in the hidden representation of credit propagation using the Shapley value decomposition method; obtaining the reward difference between the baseline and observed states for each feature dimension using the Shapley value decomposition method; and allocating the reward difference according to the feature dimension to obtain the marginal causal contribution of each dimension.

[0065] It should be noted that the Shapley value decomposition method is a statistical method based on cooperative game theory. It quantifies the independent contribution of each feature to the dependent variable by obtaining the average marginal contribution of each feature to the prediction result in all possible combinations of features.

[0066] The marginal causal contribution, historical backtracking index, and dynamic causal margin weight matrix are normalized to generate the time-series credit allocation coefficient, expressed as follows: ; in, Indicates the lag step size The corresponding time-series credit allocation coefficient, The index representing the lag step size, Indicates the lag step size The corresponding marginal causal contribution, This represents the credit decay rate coefficient. This indicates the total number of steps in the history backtracking window. This indicates that the summation traversal index variable is used. Indicates the lag step size as The marginal causal contribution of historical control actions. This represents a numerically stable regularization constant.

[0067] It should be noted that, It is obtained by exponentially fitting the temperature field response delay distribution under historical operating conditions and calibrating it based on the correspondence between the thermal inertia time constant and the causal contribution decay rate.

[0068] The specific process includes: using the product of the marginal causal contribution corresponding to the lag step length and the exponential decay term of the dynamic causal weight matrix under the lag step length as the numerator; summing the products of the marginal causal contribution of all historical control actions corresponding to the lag step length and the exponential decay term of the dynamic causal weight matrix under the lag step length in the denominator, and superimposing a numerical stability regularization constant; and calculating the time-series credit allocation coefficient corresponding to the lag step length by the ratio of the numerator and denominator, thus completing the normalization calculation of the marginal causal contribution, historical backtracking index, and dynamic causal weight matrix.

[0069] like Figure 6 The distribution diagram of time-series credit allocation coefficients shows the contribution distribution of historical control actions to the current temperature field transition under different lag steps. Among them, control group A, which did not use modal gating functions, construct dynamic physical state vectors, calculate dynamic causal edge weight matrices, or generate temporal credit allocation coefficients, only evaluated the contribution of historical actions based on conventional state inputs and conventional temporal difference methods. Therefore, the curve was relatively flat overall, indicating that the contribution of historical actions was evenly distributed across multiple lag steps, making it difficult to highlight the key historical actions that truly played a major role. Although control group B used modal gating functions, calculated convective heat transfer weights and radiative heat transfer weights, and constructed dynamic physical state vectors to make the state representation closer to the real thermal process, it could only improve the ability to identify key historical actions to a certain extent because it did not further combine dynamic causal edge weight matrices, attention-gated graph convolution, and temporal credit allocation coefficient generation mechanisms. It was still difficult to accurately reflect the true causal contribution of each historical action under conditions of strong coupling in multiple regions and large thermal lag. The experimental group, on the other hand, used dynamic physical state vectors, dynamic causal edge weight matrices, and temporal credit allocation coefficient generation mechanisms simultaneously, resulting in a more concentrated peak distribution in the key lag interval of the curve. This shows that the present invention can more accurately attribute the current temperature field transition to the historical power action that truly plays a dominant role.

[0070] S4. Based on the time-series credit allocation coefficient, the historical control cycle is weighted and integrated to obtain the modified advantage function. The modified advantage function is then backpropagated to obtain the policy network gradient and the value network gradient.

[0071] By aligning the time-series credit allocation coefficients with the time-series difference errors corresponding to the historical control periods, a credit-weighted time-series difference sequence is obtained.

[0072] The specific process includes matching and aligning the time-series credit allocation coefficients point-by-point with the time-series difference errors corresponding to the historical control cycles according to the time index, ensuring that the time-series credit allocation coefficients of each lag step accurately correspond to the time-series difference error values ​​at the same time. The time-series credit allocation coefficients are multiplied by the time-series difference errors through scalar multiplication to obtain the weighted error components of each historical time after causal weight correction. All weighted error components are arranged and combined in chronological order to form a credit-weighted time-series difference sequence containing complete historical backtracking window information.

[0073] The modified advantage function is obtained by weighted integral fusion of the credit-weighted time-series difference sequences.

[0074] The specific process includes performing a weighted integral fusion operation in the continuous time domain on the credit-weighted time-series difference sequence, transforming the discretely distributed causal correction error components into a cumulative advantage evaluation scalar, using the Riemann summation method to sum the values ​​at each time step in the credit-weighted time-series difference sequence and combining it with the control cycle step size processing to eliminate time-series fragmentation noise interference caused by large thermal inertia, integrating the comprehensive contribution effect of multiple historical actions on the current temperature field state transition, generating a corrected advantage function that reflects the superiority or inferiority of the true strategy, and accurately quantifying the expected return gain of the current heating strategy relative to the benchmark strategy.

[0075] It should be noted that the modified advantage function refers to the advantage assessment quantity formed by weighted integration of historical time-series difference errors under the action of time-series credit allocation coefficients. The modified advantage function is used to more accurately characterize the contribution of each historical control action to the current control effect and to provide a basis for the calculation of policy network gradient and value network gradient.

[0076] The modified advantage function is used to constrain the policy distribution corresponding to the historical time series difference error and historical control actions, and the value network loss function and policy network loss function are constructed.

[0077] The specific process includes using a modified advantage function to constrain the policy distribution corresponding to historical time-series difference errors and historical control actions; using the modified advantage function as a weighting factor to construct the value network loss function by applying it to the squared term of the deviation between the value network's predicted value and the actual return; enabling the value network to preferentially fit the state value assessment at times with high causal contribution; and using the modified advantage function as a gradient scaling coefficient to construct the policy network loss function by applying it to the log-likelihood term of the policy network's output action probability. This guides the policy network to update the probability distribution towards high-advantage actions, suppresses the interference of low-causal contribution actions on gradient updates, and ensures that both the value network loss function and the policy network loss function reflect the true causal logic under the physical heat transfer mechanism.

[0078] The value network gradient is obtained by backpropagating through the value network loss function, and the policy network gradient is obtained by backpropagating through the policy network loss function.

[0079] The specific process includes performing backpropagation on the value network using the value network loss function, obtaining the partial derivatives of each weight parameter in the value network with respect to the value network loss function layer by layer using the chain rule, and obtaining the value network gradient representing the direction of value assessment error reduction. At the same time, performing backpropagation on the policy network using the policy network loss function, obtaining the partial derivatives of each weight parameter in the policy network with respect to the policy network loss function layer by layer using the chain rule, and obtaining the policy network gradient representing the direction of policy distribution optimization. The value network gradient is used to guide the update of value network parameters to reduce state value prediction bias, and the policy network gradient is used to guide the update of policy network parameters to increase the selection probability of high-dominance actions. Together, they constitute the core optimization signal of the reinforcement learning agent in the vacuum furnace heating control task.

[0080] S5. Utilize the gradients of the policy network and the value network to perform optimization reasoning on the decision output distribution, generate the original heating strategy for multiple temperature zones, and perform safety projection processing through the control barrier function to generate a safe power control command.

[0081] A joint state representation is constructed based on dynamic physical state vectors, graph embedding feature vectors, and dynamic causal edge weight matrices. The distribution parameters are then mapped by combining the gradients of the policy network and the value network to obtain the initial parameters of the decision output distribution.

[0082] The specific process includes concatenating the dynamic physical state vector, graph embedding feature vector, and dynamic causal edge weight matrix along the feature dimension to generate a joint state representation. This joint state representation is then input into a policy network, where the policy network gradient is obtained through a fully connected layer. The joint state representation is then input into a value network, where the value network gradient is obtained through a fully connected layer. The policy network gradient and the value network gradient are then weighted and summed according to preset weights. The weighted summed gradient is then input into a distribution parameter mapping layer. A linear transformation in the distribution parameter mapping layer outputs the mean and log-variance parameters of the decision output distribution. Finally, the mean and log-variance parameters are combined to form the initial parameters of the decision output distribution.

[0083] It should be noted that the preset weights are preset based on the variance ratio of the policy network gradient and the value network gradient.

[0084] Based on the initial parameters of the decision output distribution, the power allocation probability corresponding to each heating zone is optimized and inferred by combining the gradient of the policy network and the gradient of the value network, so as to obtain the target heating allocation vector of the multi-temperature zone.

[0085] The specific process includes: using the initial parameters of the decision output distribution as a benchmark, the policy network gradient determines the adjustment direction of the power allocation probability by obtaining the influence direction of the power allocation probability of each heating zone on the decision output distribution; the value network gradient determines the adjustment magnitude of the power allocation probability by evaluating the rate of change of the value function corresponding to the power allocation probability of each heating zone; combining the adjustment direction determined by the policy network gradient with the adjustment magnitude determined by the value network gradient, the power allocation probability corresponding to each heating zone is iteratively updated; in each iteration, the policy network gradient and the value network gradient are re-obtained, and the power allocation probability is continuously optimized until the change in the power allocation probability is less than the convergence threshold, and the multi-temperature zone target heating allocation vector containing the optimal power allocation ratio of each heating zone is output.

[0086] It should be noted that the convergence threshold is set by combining the accuracy requirements of the initial parameters of the decision output distribution with the numerical stability of the value network gradient; an exemplary value range is 10. -5 Up to 10 -3 .

[0087] By combining the target heating allocation vector of the multi-temperature zone with the actual power distribution of the previous control cycle, smooth migration decoding is performed to obtain the original heating strategy of the multi-temperature zone.

[0088] The specific process includes matching the target allocation ratio of each heating zone in the multi-temperature zone target heating allocation vector with the actual power value of each heating zone in the actual power distribution of the previous control cycle; obtaining the transition power value between the multi-temperature zone target heating allocation vector and the actual power distribution of the previous control cycle through linear interpolation; using the transition power value as an intermediate parameter for smooth migration decoding to constrain the power adjustment range of each heating zone, avoiding sudden changes in power allocation; and combining the power values ​​of each heating zone after smooth migration decoding to form the original multi-temperature zone heating strategy.

[0089] Based on the original heating strategy for multiple temperature zones, the current temperature status of multiple temperature zones, and the dynamic causal edge weight matrix, the predicted temperature response for the next control cycle is constructed.

[0090] The specific process includes taking the power value of each heating zone in the original multi-temperature zone heating strategy as the input excitation, combining the real-time temperature value of each temperature zone in the current multi-temperature zone temperature state, using the causal influence weights between each heating zone in the dynamic causal edge weight matrix, obtaining the marginal contribution of the power value of each heating zone to the temperature change of other temperature zones, and using the superposition result of the real-time temperature value of each temperature zone and the corresponding marginal contribution as the temperature response prediction quantity for the next control cycle.

[0091] Based on the temperature response prediction, a set of control barrier function constraints is constructed by combining the upper limit of temperature in each heating zone, the upper limit of temperature difference between adjacent temperature zones, and the power change rate constraint. The original heating strategy of the multi-temperature zone is then subjected to safety projection processing using the set of control barrier function constraints to obtain the candidate vector of safe power for the multi-temperature zone.

[0092] The specific process includes: calculating the difference between the predicted temperature value of each heating zone in the temperature response prediction and the upper limit of the temperature of each heating zone; generating a corresponding temperature safety constraint term when the difference is less than zero; obtaining the difference between the upper limit of the temperature difference between adjacent temperature zones and the predicted temperature difference between adjacent heating zones in the temperature response prediction; generating a corresponding temperature difference safety constraint term when the difference is less than zero; comparing the difference between the power value of each heating zone in the original multi-temperature zone heating strategy and the power value of the previous control cycle with the power change rate constraint to generate a corresponding power safety constraint term; combining the temperature safety constraint term, the temperature difference safety constraint term, and the power safety constraint term to form a control barrier function constraint set; using a quadratic programming method to project the power value of each heating zone in the original multi-temperature zone heating strategy onto the safe area defined by the control barrier function constraint set; and combining the projected power values ​​of each heating zone to form a multi-temperature zone safe power candidate vector.

[0093] It should be noted that the control barrier function constraint set refers to a set of constraint conditions composed of multiple safety constraints, which are used to limit the safe power control command to be within the allowable safe range. The control barrier function constraint set usually includes upper temperature limit constraints for each heating zone, temperature difference constraints between adjacent temperature zones, power change rate constraints, and upper and lower power limit constraints. Its function is to avoid over-temperature, thermal imbalance, and power abrupt changes while keeping the control objective achievable.

[0094] Based on the candidate vector of safe power for multiple temperature zones, boundary correction is performed by combining the upper and lower limits of power for each heating zone and the power smoothing constraints of adjacent control cycles to obtain the safe power control command.

[0095] The specific process includes comparing the power value of each heating zone in the multi-temperature zone safe power candidate vector with the upper and lower limits of the power of each heating zone. When the power value is lower than the power limit, the power value is corrected to the lower limit. When the power value is higher than the upper limit, the power value is corrected to the upper limit. The difference between the power value of each heating zone in the multi-temperature zone safe power candidate vector and the power value of the corresponding heating zone in the previous control cycle is obtained. The difference is compared with the power smoothing constraint of the adjacent control cycle. When the difference is greater than the power smoothing constraint of the adjacent control cycle, the power value is adjusted to the sum of the power value of the previous control cycle and the power smoothing constraint of the adjacent control cycle. The power values ​​of each heating zone after boundary correction are combined to form a safe power control command.

[0096] S6. Execute the safe power control command and collect the actual temperature data for the next cycle, calculate the prediction residual, and perform incremental fine-tuning of the causal credit assessment network when the residual exceeds the limit.

[0097] Execute the safety power control command and collect the actual temperature data of each heating zone in the next control cycle.

[0098] The specific process includes transmitting the power values ​​of each heating zone in the safety power control command to the corresponding heating equipment, controlling the heating equipment to perform heating operations according to the power values ​​of each heating zone, and at the beginning of the next control cycle, using temperature sensors to monitor the temperature of each heating zone in real time, converting the temperature signals of each heating zone collected by the temperature sensors into digital signals, and using the converted digital signals as the actual temperature data of each heating zone in the next control cycle.

[0099] Based on the comparison between actual temperature data and the predicted temperature response of the corresponding control cycle, the predicted residual for each heating zone is calculated, expressed as: ; in, Indicates the first Predicted residuals for the heating zone This indicates the length of the sliding window for residual evaluation. This indicates that the index variable is traversed at each sampling time. Indicates the first The heating zone is within the sliding window. The actual temperature measurement value at the sampling time. Indicates the first The heating zone is within the sliding window. Predicted temperature response at the sampling time.

[0100] The specific process includes, for each heating zone, calculating the difference between the actual temperature measurement value of the heating zone at each sampling time within the difference evaluation sliding window length in the actual temperature data and the predicted temperature response value of the heating zone at each sampling time within the difference evaluation sliding window length in the predicted temperature response of the corresponding control cycle, squaring the differences at all sampling times and summing them, multiplying the summing result by the reciprocal of the difference evaluation sliding window length, and then taking the square root of the product to obtain the predicted residual weight of the heating zone. The above calculation process is repeated until all heating zones are traversed, and the predicted residual weight of each heating zone is output.

[0101] The predicted residuals and residual thresholds are used to determine if they exceed the limit. When the predicted residuals exceed the limit, an incremental fine-tuning sample corresponding to the current control period is constructed, and the local parameters of the causal credit assessment network are incrementally fine-tuned based on the incremental fine-tuning sample.

[0102] The specific process includes comparing the predicted residual with the residual threshold one by one. When the predicted residual of any heating zone is greater than the residual threshold, it is determined that the predicted residual exceeds the limit. The actual temperature data, safe power control command and predicted temperature response of each heating zone in the current control cycle are collected. The collected data are combined to form the incremental fine-tuning sample corresponding to the current control cycle. The incremental fine-tuning sample is input into the causal credit assessment network. The local parameters related to the heating zone with the predicted residual exceeding the limit in the causal credit assessment network are incrementally adjusted to complete the incremental fine-tuning of the local parameters of the causal credit assessment network.

[0103] It should be noted that the residual threshold is usually preset based on the standard deviation of historical temperature data or the maximum allowable temperature deviation, with an exemplary range of 1% to 5% of the actual temperature range.

[0104] In summary, this invention achieves real-time quantification of the proportion of heat transfer modes and high-fidelity physical reconstruction of the state space under continuous gas extraction and venting conditions in a vacuum furnace by using modal gating functions to calculate the weights of convection and radiation heat transfer and constructing dynamic physical state vectors. This results in faithful state characterization and improved temperature field tracking accuracy under variable vacuum conditions, while effectively suppressing control command oscillations and frequent triggering of safety constraints. Furthermore, by using dynamic causal edge weight calculation and attention graph convolution constraints to generate temporal credit allocation coefficients, this invention achieves true causal separation of historical power actions from current temperature field transitions in multi-region strongly coupled and large thermal hysteresis systems.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning, characterized in that, include: The smooth vacuum degree sequence and multi-temperature zone temperature sequence of the vacuum furnace are collected, and normalized mapping is performed through the modal gating function to obtain the convective heat transfer weight and the radiative heat transfer weight. Based on the convective heat transfer weight and the radiative heat transfer weight, the heat flow characteristics are constructed to obtain the dynamic physical state vector. The dynamic causal edge weight matrix is ​​calculated using dynamic physical state vectors, and multi-round node feature aggregation is performed through attention-gated graph convolution mechanism to obtain graph embedding feature vectors; The graph embedding feature vector, dynamic causal edge weight matrix and historical action window are input into the causal credit assessment network, and message passing is constrained by the graph attention mechanism to generate time-series credit allocation coefficients. Based on the time-series credit allocation coefficient, the historical control cycle is weighted and integrated to obtain the modified advantage function. The modified advantage function is then backpropagated to obtain the policy network gradient and the value network gradient. The policy network gradient and value network gradient are used to optimize the distribution of decision outputs and generate multi-temperature zone original heating strategies. The safety projection is then performed through the control barrier function to generate safe power control commands. The system executes safe power control commands and collects actual temperature data for the next cycle, calculates the prediction residuals, and performs incremental fine-tuning of the causal credit assessment network when the residuals exceed the limits.

2. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The specific steps for obtaining the convective heat transfer weight and the radiative heat transfer weight are as follows: The raw vacuum level data and raw temperature data of each heating zone of the vacuum furnace are collected, and the raw vacuum level data are smoothed and time-series aligned to obtain a smoothed vacuum level sequence and a multi-temperature zone temperature sequence. Based on the smooth vacuum degree sequence, the multi-temperature zone temperature sequence and the preset initial heat transfer parameters, the intensity of convective candidate heat flux and the intensity of radiative candidate heat flux are calculated. Based on the convective candidate heat flux intensity and the radiative candidate heat flux intensity, normalization mapping is performed through a modal gating function to obtain the convective heat transfer weight and the radiative heat transfer weight.

3. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The specific steps to obtain the dynamic physical state vector are as follows: By linearly combining the heat transfer states of each heating zone with the convective heat transfer weight and the radiative heat transfer weight, the heat flow characteristics corresponding to each heating zone are constructed. By assembling the heat flow characteristics, smoothed vacuum degree sequence, and multi-temperature zone temperature sequence, a dynamic physical state vector is obtained.

4. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 3, characterized in that, The specific steps for calculating the dynamic causal edge weight matrix using dynamic physical state vectors are as follows: Extract multi-dimensional feature vectors of each node from the dynamic physical state vector and the temperature gradient difference between adjacent regions; Based on the multidimensional feature vectors of each node and the temperature gradient difference between adjacent regions, and combined with the convective heat transfer weight and the radiative heat transfer weight, the dynamic causal edge weight matrix is ​​calculated through the modal causal edge weight function.

5. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The specific steps for obtaining the graph embedding feature vector are as follows: Based on the dynamic causal edge weight matrix and the multidimensional feature sub-vectors of each node, multiple rounds of node feature aggregation are performed through the attention-gated graph convolution mechanism to obtain the iterative node embedding set. Global pooling and dual-channel fusion, along with row normalization probability mapping, are applied to the node embedding set to obtain the graph embedding feature vector.

6. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1 or 5, characterized in that, The specific steps for generating the time-series credit allocation coefficients are as follows: The graph embedding feature vector, dynamic causal edge weight matrix and historical action window are input into the causal credit assessment network for time-series correlation encoding to generate a graph conditional action event sequence. The key historical action set is then extracted from the graph conditional action event sequence to obtain the historical backtracking index. A temporal credit graph of actions is constructed using historical backtracking indexes, and a graph attention mechanism is used to constrain the message passing of nodes in the temporal credit graph of actions using a dynamic causal edge weight matrix, thereby obtaining a hidden representation of credit propagation. By performing prefix reward decomposition on the hidden representation of credit propagation, the marginal causal contribution is obtained; The marginal causal contribution, historical backtracking index, and dynamic causal weight matrix are normalized and calculated to generate time-series credit allocation coefficients.

7. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The specific steps for obtaining the policy network gradient and the value network gradient are as follows: By aligning the time-series credit allocation coefficients with the time-series difference errors corresponding to the historical control periods, a credit-weighted time-series difference sequence is obtained. The modified advantage function is obtained by weighted integral fusion of the credit-weighted time-series difference sequences. The modified advantage function is used to constrain the policy distribution corresponding to the historical time series difference error and the historical control action, and the value network loss function and policy network loss function are constructed. The value network gradient is obtained by backpropagating through the value network loss function, and the policy network gradient is obtained by backpropagating through the policy network loss function.

8. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1, characterized in that, The specific steps for generating the original multi-temperature zone heating strategy are as follows: A joint state representation is constructed based on dynamic physical state vectors, graph embedding feature vectors, and dynamic causal edge weight matrices. The distribution parameters are then mapped by combining the gradients of the policy network and the value network to obtain the initial parameters of the decision output distribution. Based on the initial parameters of the decision output distribution, the power allocation probability corresponding to each heating zone is optimized and inferred by combining the gradient of the policy network and the gradient of the value network, so as to obtain the target heating allocation vector of the multi-temperature zone. By combining the target heating allocation vector of the multi-temperature zone with the actual power distribution of the previous control cycle, smooth migration decoding is performed to obtain the original heating strategy of the multi-temperature zone.

9. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 1 or 8, characterized in that, The specific steps for generating the safe power control command are as follows: Based on the original multi-temperature zone heating strategy, the current temperature state of the multi-temperature zone, and the dynamic causal edge weight matrix, the temperature response prediction for the next control cycle is constructed. Based on the temperature response prediction, a set of control barrier function constraints is constructed by combining the upper limit of temperature in each heating zone, the upper limit of temperature difference between adjacent temperature zones, and the power change rate constraint. The original heating strategy of the multi-temperature zone is then subjected to safety projection processing using the set of control barrier function constraints to obtain the candidate vector of safe power for the multi-temperature zone. Based on the candidate vector of safe power for multiple temperature zones, boundary correction is performed by combining the upper and lower limits of power for each heating zone and the power smoothing constraints of adjacent control cycles to obtain the safe power control command.

10. The method for dynamically adjusting the heating strategy of a vacuum furnace based on reinforcement learning as described in claim 9, characterized in that, The process involves executing safe power control commands, collecting actual temperature data for the next cycle, calculating prediction residuals, and incrementally fine-tuning the causal credit assessment network when the residuals exceed limits. The specific steps are as follows: Execute the safety power control command and collect the actual temperature data of each heating zone in the next control cycle; The predicted residuals for each heating zone are calculated by comparing the actual temperature data with the predicted temperature response of the corresponding control cycle. The predicted residuals and residual thresholds are used to determine if they exceed the limit. When the predicted residuals exceed the limit, an incremental fine-tuning sample corresponding to the current control period is constructed, and the local parameters of the causal credit assessment network are incrementally fine-tuned based on the incremental fine-tuning sample.

Images