An industrial process quality prediction method based on variable time delay estimation

Abstract

The application provides an industrial process quality prediction method based on variable time delay estimation, and belongs to the technical field of industrial artificial intelligence. The method comprises the following steps: acquiring a real-time process variable sequence in an industrial process; inputting the real-time process variable sequence into a trained high-level DQN network to obtain an interval range of a corresponding time delay parameter; inputting the interval range of the corresponding time delay parameter into a trained low-level DQN network to obtain a current time delay parameter configuration; performing time alignment and time sequence sample reconstruction on the real-time process variable sequence according to the current time delay parameter configuration; and inputting the reconstructed sequence into a trained quality index prediction model to obtain an end point quality index prediction result. The method improves the efficiency of variable time delay estimation, effectively overcomes the time lag linearity in a long industrial process through time delay estimation, and improves the accuracy of end point quality prediction.

Description

Technical Field

[0001] This application belongs to the field of industrial artificial intelligence technology and relates to an industrial process quality prediction method based on variable time delay estimation. Background Technology

[0002] Complex industrial processes are typically characterized by long production flows, high levels of equipment interconnection, and continuous material flow. Influenced by factors such as the spatial distribution of equipment, the paths of material transport, and differences in reaction mechanisms, the impact of different process variables on final quality indicators does not occur synchronously, but rather exhibits varying degrees of time lag. This is especially true in continuous production systems composed of multi-stage pipelines, heat exchangers, and reaction units, where sensor installation locations are dispersed. Process data collected at the same time stamp often corresponds to process states at different time stages, thus disrupting the temporal relationship between input variables and quality indicators. This dynamic time lag variation, changing with operating conditions, constitutes one of the key challenges in modeling and predicting complex industrial processes. Taking the leaching process in Bayer alumina production as an example, the slurry flows continuously through multiple preheating, heating, and leaching reaction units. The impact of process variables such as temperature, flow rate, and pressure at different locations on final indicators exhibits significant non-uniform lag characteristics. When production load, material flow rate, or operating conditions change, this time lag relationship also changes, displaying significant variable time lag characteristics. However, existing industrial data acquisition systems typically sample variables at a uniform timestamp, which fails to reflect the true causal time-series relationships. This severely limits the prediction accuracy and generalization performance of models built on historical data. Traditional methods based on fixed time delays or simple correlation analysis struggle to accurately characterize the dynamic time-delay characteristics between multiple variables under complex operating conditions, thus affecting the accurate assessment of endpoint quality indicators and the effective judgment of process operating status.

[0003] Therefore, in order to effectively estimate the variable time-delay relationship between multiple variables and endpoint quality indicators during the operation of complex industrial processes, and thereby improve the accuracy and stability of prediction results, this invention proposes an industrial process quality prediction method based on variable time-delay estimation, which has high industrial application value. Summary of the Invention To address the shortcomings of existing technologies, this application provides a method for predicting the quality of industrial processes based on variable time delay estimation.

[0004] Firstly, this application proposes an industrial process quality prediction method based on variable time delay estimation, comprising: Obtain real-time process variable sequences in industrial processes; By inputting the real-time process variable sequence into the trained high-level DQN network, the range of the corresponding time delay parameters can be obtained. Input the range of the corresponding time delay parameters into the trained low-level DQN network to obtain the current time delay parameter configuration; Based on the current time delay parameter configuration, perform time alignment and time series sample reconstruction on the real-time process variable sequence; The reconstructed sequence is input into the trained quality index prediction model to obtain the final endpoint quality index prediction result.

[0005] An industrial process quality prediction method based on variable time delay estimation further includes: constructing a training set, using the training set to train a high-level DQN network, a low-level DQN network, and a quality index prediction model, respectively, to obtain the trained high-level DQN network, the trained low-level DQN network, and the trained quality index prediction model. The construction of the training set includes: collecting a sequence of multidimensional historical process variables that have a causal relationship with the endpoint quality index during industrial operation and the corresponding endpoint quality index data, and setting a one-to-one time delay parameter for each historical process variable. The time delay parameter is used to characterize the time delay step of the corresponding historical process variable relative to the endpoint quality index data.

[0006] The process involves training a high-level DQN network, a low-level DQN network, and a quality indicator prediction model using a training set, respectively, to obtain the trained high-level DQN network, the trained low-level DQN network, and the trained quality indicator prediction model, including: The interval range of time delay parameters corresponding to each historical process variable in the training set is coarsely filtered by a high-level DQN network to obtain the interval range of the filtered time delay parameters. By using a low-level DQN network, the time delay parameters of each historical process variable are finely filtered within the range of the filtered time delay parameters to obtain the historical time delay parameter configuration. Input the historical time delay parameters into the low-level agent model to obtain the predicted values ​​of the low-level endpoint quality indicators; Calculate the low-level prediction error between the predicted value of the endpoint quality index and the actual endpoint quality index. Construct a first reinforcement learning reward signal based on the low-level prediction error. Update the network parameters of the low-level DQN network using the first reinforcement learning reward signal to obtain the trained low-level DQN network. The trained low-level DQN network is used to further refine the time delay parameters of each historical process variable within the range of the selected time delay parameters to obtain updated time delay parameters. Based on the updated time delay parameters, the time series of each historical process variable in the training set are differentiated and time-aligned with the corresponding time of the endpoint quality index data as the benchmark, and reconstructed using a time window of preset length L to obtain the process variable series used for quality prediction. By inputting the process variable sequence into the quality indicator prediction model, the predicted values ​​of the final quality indicators at the senior management level are obtained. Calculate the high-level prediction error between the high-level predicted value of the endpoint quality index and the actual endpoint quality index, and use the high-level prediction error to update the quality index prediction model to obtain the trained quality index prediction model. A second reinforcement learning reward signal is constructed based on the high-level prediction error. The network parameters of the high-level DQN network are updated using the second reinforcement learning reward signal to obtain the trained high-level DQN network.

[0007] The quality indicator prediction model includes: a forward TCN module, a reverse TCN module, a high-level prediction task head, and a low-level proxy task head. The process variable sequence is input into the forward TCN module to obtain forward time-series features; the process variable sequence is input into the reverse TCN module to obtain reverse time-series features; the forward and reverse time-series features are fused to obtain shared time-series features; the shared time-series features are input into the high-level prediction task head to obtain the high-level endpoint quality indicator prediction value; and the shared time-series features are input into the low-level proxy task head to obtain the low-level endpoint quality indicator prediction result.

[0008] The step of inputting the reconstructed sequence into the trained quality index prediction model to obtain the final endpoint quality index prediction result includes: inputting the reconstructed sequence into the trained quality index prediction model, and using the low-level endpoint quality index prediction result as the final endpoint quality index prediction result.

[0009] By inputting the real-time process variable sequence into the trained high-level DQN network, the corresponding time delay parameter range is obtained, including: Set a one-to-one time delay parameter for each historical process variable; The real-time process variable sequence is used as the state space of the Markov decision process; The action space of a Markov decision process is represented as low-lag actions, medium-lag actions, and high-lag actions. Establish a mapping relationship between the action space and the time delay parameters; The real-time process variable sequence is input into the trained high-level DQN network to obtain the first Q value of each candidate action. For each process variable in the real-time process variables, select the first target action corresponding to the largest first Q value; For the first target action, the range of the corresponding time delay parameters is obtained based on the mapping relationship between the action space and the time delay.

[0010] The step of inputting the corresponding time delay parameter range into the trained low-level DQN network to obtain the current time delay parameter configuration includes: The range of the corresponding time delay parameter is used as the state parameter in the low-level decision-making. Set the fine-tuning step size as the action space in low-level decision-making; Input the interval range of the corresponding time delay parameter into the trained low-level DQN network to obtain the second Q value of each process variable when performing the decrease, maintain, and increase actions respectively; For each process variable, select the second objective action corresponding to the largest Q value; Based on the state parameters in the lower-level decision-making process and the second objective action, the current time delay parameter configuration is obtained.

[0011] The step of time alignment and time series sample reconstruction of the real-time process variable sequence based on the current time delay parameter configuration includes: Based on the endpoint quality index data, the i-th process variable is shifted according to the current time delay parameter configuration to obtain the reconstructed process variable, as shown in the following expression: ; in, Let i be the reconstructed process variable at time t. Configure the current time delay parameter for the i-th process variable. Let i be the i-th process variable.

[0012] A first reinforcement learning reward signal is constructed based on the low-level prediction error, and a second reinforcement learning reward signal is constructed based on the high-level prediction error, including: ; ; in, This is a second reinforcement learning reward signal. As the first reinforcement learning reward signal, This is due to high-level forecasting errors. The low-level prediction error is represented by N, where N is the total number of samples in the training set. Let j be the true endpoint quality index for the j-th sample. Let be the predicted value of the endpoint quality index for the high-level or low-level sample j. H Indicates high-level personnel. L Indicates the lower level.

[0013] Beneficial technical effects: This application discloses an industrial process quality prediction method based on variable time delay estimation. By introducing a hierarchical reinforcement learning mechanism, the variable time delay between multiple variables and the endpoint quality index in the industrial process is estimated from coarse to fine, which improves the efficiency of variable time delay estimation. Furthermore, the time delay estimation effectively overcomes the linearity of time lag in long industrial processes, improves the accuracy of endpoint quality prediction, and enhances the online application capability and engineering applicability of the method in real industrial scenarios. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating an industrial process quality prediction method based on variable time delay estimation, provided for an embodiment of this application; Figure 2 This is a schematic diagram of the alumina leaching process provided in the embodiments of this application; Figure 3 A framework diagram of a hierarchical reinforcement learning variable time delay estimation method provided in an embodiment of the present invention; Figure 4 A framework diagram of the feature sharing mechanism provided in the embodiments of the present invention; Figure 5 This is a schematic diagram of the prediction results of the endpoint quality index provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0017] Example 1 This embodiment proposes an industrial process quality prediction method based on variable time delay estimation, such as... Figure 1 As shown, it includes: Step S1: Obtain the real-time process variable sequence in the industrial process; In this embodiment, multidimensional real-time process variables that have a causal relationship with the endpoint quality index are collected during industrial operation. The multidimensional real-time process variables are preprocessed and normalized, and the normalized real-time process variables are used as the input for the subsequent step S2.

[0018] Specifically, this embodiment selects a representative leaching process in the alumina production process, such as... Figure 2 The illustration shows the application of an industrial process quality prediction method based on variable time delay estimation in a real industrial environment. It should be noted that the following embodiments are only used to explain the technical concept and implementation path of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0019] In continuous industrial production, the leaching process typically consists of multiple series-connected heating, reaction, and material conveying units. Process variables are distributed across different spaces along the production line, and sampling data from different measuring points exhibit significant time lags. Taking the leaching process as an example, multiple process variables jointly participate in and influence the formation of the final quality indicators, including but not limited to material flow rate, temperature, pressure, liquid level, and chemical composition-related variables. These variables have strong nonlinear coupling relationships, and their impact on the final quality indicators does not occur simultaneously but is accompanied by varying degrees of time lag. Due to factors such as production load adjustments, operating condition switching, and changes in equipment operating status, these time lag relationships are not fixed but exhibit dynamic changes during the production process, forming a typical variable time lag problem. However, existing industrial control systems typically use a uniform timestamp to sample each process variable, resulting in a lack of strict alignment between multivariate process data and the final quality indicators in the time dimension, thus disrupting the true causal relationship between variables. This time mismatch problem makes it difficult for traditional prediction methods based on fixed time lags or static modeling assumptions to maintain stable prediction performance under complex operating conditions. Furthermore, in continuous production environments, some endpoint quality indicators are difficult to acquire online at high frequency and with high accuracy due to limitations in measurement conditions, and their variation characteristics can often only be estimated indirectly. This places higher demands on data-driven prediction models, which not only need to have the ability to model complex nonlinear relationships, but also need to be able to dynamically adapt to the time lag characteristics between multiple variables and endpoint quality indicators as operating conditions change.

[0020] To address the aforementioned issues, this embodiment proposes an industrial process quality prediction method based on variable time delay estimation. By modeling the multivariable variable time delay estimation process as a hierarchical decision problem, a collaborative mechanism between a high-level DQN network and a low-level DQN network is introduced to achieve online optimization of the time delay parameters from coarse to fine. Simultaneously, by combining it with a quality index prediction model, the computational overhead of model training and evaluation is significantly reduced while ensuring prediction accuracy, thereby improving the method's real-time performance and engineering applicability in actual industrial processes.

[0021] Step S2: Input the real-time process variable sequence into the trained high-level DQN network to obtain the corresponding time delay parameter range; By inputting the real-time process variable sequence into the trained high-level DQN network, the corresponding time delay parameter range is obtained, including: Step S2.1: Set a one-to-one time delay parameter for each historical process variable, as shown in the following expression: ; in, To indicate the first i The time lag step of each process variable relative to the endpoint quality indicator. nThe number of process variables, To indicate the first n The time lag step of each process variable relative to the endpoint quality indicator; Step S2.2: Treat the real-time process variable sequence as the state space of the Markov decision process, as shown in the following expression: ; in, Indicates the first The observed values ​​of a real-time process variable at the current moment. For state space, Indicates the first n The observed values ​​of a real-time process variable at the current moment; Understandably, to ensure that each process variable has only one corresponding time delay parameter, this embodiment uses a fixed index to establish a one-to-one correspondence between the time delay parameter and the process variable. d i The corresponding one is x i The time delay parameter; To prevent excessive time spent on exploration in the early stages of reinforcement learning, the time delay parameter is first initialized to... The high-level decision-making process divides the time delay parameter into three levels—low, medium, and high—for coarse-grained selection, in order to reduce the search space for variable time delays and increase search efficiency. Step S2.3: Represent the action space of the Markov decision process as low-lag actions, medium-lag actions, and high-lag actions, as shown in the following expressions: ; in, This represents the action space, where 0 indicates a low-latency action, 1 indicates a medium-latency action, and 2 indicates a high-latency action. Step S2.3: Establish the mapping relationship between the action space and the time delay parameters, as shown in the following expression: .

[0022] In this embodiment, in order to transform abstract actions into specific physical time delays, action labels and time delay parameters are established. The mapping relationship between them.

[0023] Step S2.4: Input the real-time process variable sequence into the trained high-level DQN network to obtain the first Q value of each candidate action; Step S2.5: For each process variable in the real-time process variables, select the first target action corresponding to the largest first Q value; In this embodiment, both the high-level DQN network and the low-level DQN network are DQN (Deep Q-Network) structures. While the high-level and low-level DQN networks are fundamentally the same, some network parameters differ due to their different tasks. This embodiment also employs an experience replay storage pool. The input to the experience replay pool is the current state, action, reward, and next state, and the output is a batch of state transition samples randomly selected from historical experience. Specifically, it stores the state transition experiences generated by the high-level and low-level DQN networks respectively, and constructs mini-batch temporal difference training samples through random sampling to update the action value function of the corresponding DQN network. Both the high-level and low-level DQN networks are existing technologies and will not be described in detail in this embodiment.

[0024] Step S2.6: For the first target action, obtain the range of the corresponding time delay parameters based on the mapping relationship between the action space and the time delay.

[0025] In this embodiment, the Markov decision process is as follows: the high-level state is composed of the normalized multidimensional process variables at the current moment; the high-level action is the lag parameter interval corresponding to the high, medium and low levels; the high-level state transition relationship is the conditional probability that the industrial process will enter the next high-level state after executing the high-level action in the current industrial process state; the high-level reward function is constructed based on the prediction error of the endpoint quality index under the high-level coarse-grained time delay configuration; the high-level discount factor is 0.95 in this embodiment.

[0026] In addition, the training process of the high-level DQN network after training requires first constructing a training set. The high-level DQN network, the low-level DQN network, and the quality index prediction model are trained using the training set to obtain the trained high-level DQN network, the trained low-level DQN network, and the trained quality index prediction model. The construction of the training set includes: collecting multi-dimensional historical process variable sequences that have a causal relationship with the endpoint quality index during industrial operation and the corresponding endpoint quality index data; setting a one-to-one time delay parameter for each historical process variable; the time delay parameter is used to characterize the time delay step of the corresponding historical process variable relative to the endpoint quality index data.

[0027] To address the characteristic that multiple process variables in industrial processes have different time lags for the endpoint quality index, and that these lags vary with operating conditions, the variable time delay estimation problem is modeled as a Markov decision process, and then optimized by introducing a hierarchical reinforcement learning structure.

[0028] The quality indicator prediction model includes: a forward TCN module, a reverse TCN module, a high-level prediction task head, and a low-level proxy task head. The process variable sequence is input into the forward TCN module to obtain forward time-series features; the process variable sequence is input into the reverse TCN module to obtain reverse time-series features; the forward and reverse time-series features are fused to obtain shared time-series features; the shared time-series features are input into the high-level prediction task head to obtain the high-level endpoint quality indicator prediction value; and the shared time-series features are input into the low-level proxy task head to obtain the low-level endpoint quality indicator prediction result.

[0029] In this embodiment, the multivariable variable time delay estimation problem is modeled as a decision process, and a hierarchical reinforcement learning structure is introduced, setting up a high-level DQN network and a low-level DQN network. The high-level network is used to make coarse-grained decisions on the time delay range corresponding to each process variable, so as to quickly narrow the search space of the variable time delay and reduce the overall decision complexity.

[0030] Within the time delay range defined by the high-level DQN network, the low-level DQN network performs fine-grained adjustments to the time delay parameters of each variable. The low-level DQN network rapidly evaluates the current time delay configuration and uses the evaluation results as feedback signals to guide the gradual optimization of the time delay parameters, thereby significantly reducing computational overhead and improving search efficiency while ensuring estimation accuracy.

[0031] Step S3: Input the range of the corresponding time delay parameters into the trained low-level DQN network to obtain the current time delay parameter configuration; In this embodiment, based on the range of the corresponding time delay parameters determined at the higher level, a lower-level DQN network is then used to perform fine-grained optimization of the time delay parameters. During the lower-level decision-making process, the state space... This represents a combination of coarse-grained time delay parameters input from a higher level.

[0032] The step of inputting the corresponding time delay parameter range into the trained low-level DQN network to obtain the current time delay parameter configuration includes: Step S3.1: Use the range of the corresponding time delay parameter as the state parameter in the low-level decision-making process, as shown in the following expression: ; in, These are state parameters in low-level decision-making. Let be the time delay parameter corresponding to the i-th process variable; Step S3.2: Set the fine-tuning step size as the action space in the low-level decision-making process, as shown in the following expression: ; in, This represents the action space in low-level decision-making; -1, 0, and 1 represent the step size for fine-tuning the current time delay parameter, respectively; where -1 indicates that the time delay parameter decreases by one sampling step, 0 indicates that it remains unchanged, and 1 indicates that it increases by one sampling step.

[0033] In this embodiment, the actions of the low-level decision-making process are used to make local adjustments to the time delay parameters.

[0034] Step S3.3: Input the interval range of the corresponding time delay parameter into the trained low-level DQN network to obtain the second Q value of each process variable when performing the decrease, maintain, and increase actions respectively; Step S3.4: For each process variable, select the second target action corresponding to the largest second Q value; Step S3.5: Based on the state parameters in the lower-level decision and the second objective action, obtain the current time delay parameter configuration, as shown in the following expression: ; in, Configure the current time delay parameter for the i-th process variable. For the second objective action, specifically, the low-level DQN network calculates the second Q value for each process variable when performing three fine-tuning actions: decrease, maintain, and increase. For each process variable, the second objective action with the largest second Q value is selected, and this second objective action is transformed into... 1, 0, or +1 time lag adjustment; where 1 indicates that the time delay parameter decreases by one sampling step, 0 indicates that it remains unchanged, and +1 indicates that it increases by one sampling step. This ensures that the time delay parameter is covered. This preset range enables fine-grained local search and optimization around a given lag scale at a higher level. In industrial applications, this range can be adjusted according to different process industrial conditions.

[0035] In this embodiment, the Markov decision process is as follows: the low-level state is composed of the candidate time delay intervals of each process variable determined by the high-level DQN network and the current time delay parameters within the candidate time delay intervals; the low-level action is to perform fine-grained adjustments on the current time delay parameters of each process variable by decreasing by one sampling step, keeping them unchanged, or increasing by one sampling step; the low-level state transition relationship is a deterministic transition relationship in which the current time delay parameter configuration is transferred to the next time delay parameter configuration according to the time delay parameter update rule after the low-level action is performed; the low-level reward function is constructed based on the prediction error of the endpoint quality index obtained by the low-level agent model under the fine-grained time delay configuration; the low-level discount factor is taken as 0.95 in this embodiment.

[0036] The process involves training a high-level DQN network, a low-level DQN network, and a quality indicator prediction model using a training set, respectively, to obtain the trained high-level DQN network, the trained low-level DQN network, and the trained quality indicator prediction model, including: Step S100: Use a high-level DQN network to coarsely filter the range of time delay parameters corresponding to each historical process variable in the training set to obtain the range of time delay parameters after filtering. Step S101: Use a low-level DQN network to perform fine-grained filtering of the time delay parameters of each historical process variable within the range of the filtered time delay parameters to obtain the historical time delay parameter configuration; Step S102: Input the historical time delay parameter configuration into the low-level agent model to obtain the predicted value of the low-level endpoint quality index; Step S103: Calculate the low-level prediction error between the predicted value of the low-level endpoint quality index and the actual endpoint quality index. Construct a first reinforcement learning reward signal based on the low-level prediction error. Update the network parameters of the low-level DQN network using the first reinforcement learning reward signal to obtain the trained low-level DQN network. Step S104: The trained low-level DQN network is used to further refine the time delay parameters of each historical process variable within the range of the selected time delay parameters to obtain the updated time delay parameters; Step S105: Based on the updated time delay parameters, the time series of each historical process variable in the training set are differentiated and time-aligned with the corresponding time of the endpoint quality index data as the benchmark, and reconstructed using a time window of preset length L to obtain the process variable sequence used for quality prediction; Step S106: Input the process variable sequence into the quality index prediction model to obtain the predicted values ​​of the final quality index at the senior management level; Step S107: Calculate the high-level prediction error between the high-level predicted value of the endpoint quality index and the actual endpoint quality index, and use the high-level prediction error to update the quality index prediction model to obtain the trained quality index prediction model. Step S108: Construct a second reinforcement learning reward signal based on the high-level prediction error, and use the second reinforcement learning reward signal to update the network parameters of the high-level DQN network to obtain the trained high-level DQN network.

[0037] Furthermore, to clarify the network parameter update process of the high-level DQN network and the low-level DQN, this embodiment sets up experience replay pools for both the high-level and low-level agents. The high-level prediction model outputs the predicted value of the endpoint quality index based on the high-level coarse-grained time delay parameters, and the low-level agent prediction model outputs the predicted value of the endpoint quality index based on the low-level fine-grained time delay parameters. The mean square error of the high-level prediction and the mean square error of the low-level prediction are calculated respectively, and their negative values ​​are used as the immediate reward of the corresponding level of DQN.

[0038] A first reinforcement learning reward signal is constructed based on the low-level prediction error, and a second reinforcement learning reward signal is constructed based on the high-level prediction error, including: ; ; in, This is a second reinforcement learning reward signal. As the first reinforcement learning reward signal, This is due to high-level forecasting errors. The low-level prediction error is represented by N, where N is the total number of samples in the training set. Let j be the true endpoint quality index for the j-th sample. Let be the predicted value of the endpoint quality index for the high-level or low-level sample j. H Indicates high-level personnel. L Indicates the lower level.

[0039] The high-level DQN and low-level DQN randomly sample training samples from their respective experience replay pools and construct the target action value function based on the immediate reward of the corresponding level. For the terminal state, the target Q-value equals the immediate reward; for the non-terminal state, the target Q-value is determined by the immediate reward and the maximum action value of the next state. ; in, The target action value corresponding to the current action performed in the current state. The target action value corresponding to the next action in the next state. For discount factor, Current status For the current action, For the next state, For the next action, The immediate reward for performing this action. Accordingly, both high-level and low-level DQN networks update their network parameters by minimizing the temporal mean square error of the difference between the current Q-value and the target Q-value, and their loss function is expressed as: ; in, Let N be the loss function for the high-level DQN network and the low-level DQN network, and let N be the number of samples in the entire training set. The mean squared error of the prediction is the value of the action performed in the current state. Therefore, the mean squared error of the prediction is used to construct the immediate rewards for high-level and low-level DQNs, while the DQN network itself updates its parameters through temporal difference mean squared error, causing the reinforcement learning policy to evolve in the direction of reducing prediction error.

[0040] Step S4: Perform time alignment and time series sample reconstruction on the real-time process variable sequence according to the current time delay parameter configuration; In this embodiment, the data is reconstructed using a time window method based on the current time delay parameter configuration, so that the input data of the prediction model is consistent with the endpoint quality index in time, thereby eliminating the time lag problem caused by uniform timestamp sampling.

[0041] The step of time alignment and time series sample reconstruction of the real-time process variable sequence based on the current time delay parameter configuration includes: Based on the endpoint quality index data, the i-th process variable is shifted according to the current time delay parameter configuration to obtain the reconstructed process variable, as shown in the following expression: ; in, Let i be the reconstructed process variable at time t. Configure the current time delay parameter for the i-th process variable. Let be the i-th process variable. Its endpoint quality index is not subject to time lag processing, ensuring consistency between input data and output index in a physical causal sense. Subsequently, a fixed-length time window is used to reconstruct the aligned process variable, resulting in a two-dimensional time-series feature matrix with a time span that can serve as the input variable for the quality index prediction model. This forms a process variable sequence sample (the process variable sequence is data continuously collected from any process variable within a time period L) used for subsequent prediction model training and online prediction.

[0042] Time series sample reconstruction, the expression is as follows: ; in, The sequence is the reconstructed sequence, and L is the window length. Let be the aligned multidimensional process variable vector at time t. This is the aligned multidimensional process variable vector corresponding to the starting time t-L+1 of the current time window.

[0043] Step S5: Input the reconstructed sequence into the trained quality index prediction model to obtain the endpoint quality index prediction result.

[0044] In this embodiment, a quality indicator prediction model is constructed to enable the high-level prediction model (i.e. Figure 3 The mid-level prediction model (BiTCN) and the low-level proxy model (i.e. Figure 3The proxy model shares a unified feature representation in the temporal feature extraction stage and completes the tasks of endpoint quality index prediction and time delay parameter evaluation respectively, so as to reduce the computational overhead caused by repeated feature extraction, improve the training efficiency in the hierarchical reinforcement learning process, and use the prediction performance of the endpoint quality index as the evaluation basis for the hierarchical reinforcement learning decision process.

[0045] The step of inputting the reconstructed sequence into the trained quality index prediction model to obtain the final endpoint quality index prediction result includes: inputting the reconstructed sequence into the trained quality index prediction model, and using the low-level endpoint quality index prediction result as the final endpoint quality index prediction result.

[0046] In this embodiment, the quality index prediction model is actually a shared-feature multi-task quality prediction model, including a forward TCN module, a backward TCN module, a high-level prediction task head, and a low-level proxy task head. The forward TCN module and the backward TCN module are shared feature extraction layers used by both task heads, and the direct inputs to both task heads are shared temporal features. The high-level prediction task head outputs the high-level endpoint quality index prediction value and uses the reconstructed sequence as input to output the final endpoint quality index prediction result. When the input to the trained quality index prediction model is the reconstructed sequence, the output of the low-level proxy task head, i.e., the low-level endpoint quality index prediction result, is used as the final endpoint quality index prediction result, such as... Figure 4 As shown, the current candidate time delay parameter configuration is evaluated by calculating the MSE or R² between the surrogate prediction value and the true quality metric, and a low-level DQN reward is constructed. The high-level prediction task head adopts a "128-dimensional fully connected layer - Dropout - single neuron output layer" structure; the low-level surrogate task head adopts a "64-dimensional fully connected layer - Dropout - 32-dimensional fully connected layer - single neuron output layer" structure; both share a shared bidirectional TCN feature extraction layer composed of forward TCN branch, backward TCN branch, global average pooling, and feature concatenation.

[0047] In this embodiment, after determining the time delay parameters and reconstructing the time window, a quality indicator prediction model for predicting the endpoint quality indicators is constructed, and the prediction performance is used as an evaluation criterion for optimizing the time delay parameters through hierarchical reinforcement learning.

[0048] Quality indicator prediction model for reconstructed process variable series Perform feature extraction; The forward TCN module, or forward temporal convolution branch, models the sequence of input process variables in chronological order to characterize the causal relationship between process variables and the endpoint quality index. ; in, For the reconstructed sequence, Forward TCN module, It represents a positive time series characteristic.

[0049] The reverse TCN module, i.e., the reverse temporal convolution branch, first reverses the process variable sequence and then performs temporal convolution modeling on this basis to compensate for feedback effects and non-unidirectional dynamic correlations that may exist in complex industrial processes. ; ; in, It is a reversed sequence. For reverse TCN modules, It is a reverse time series feature.

[0050] The temporal feature representations of the multidimensional industrial process variables output by the forward temporal convolution branch and the backward temporal convolution branch are fused to obtain a shared temporal feature representation: ; in, H To share temporal characteristics.

[0051] Based on the aforementioned shared feature representation, a prediction output layer is constructed to perform regression prediction on the endpoint quality index. This prediction output layer includes a high-level prediction task header and a low-level proxy task header, and its prediction result is expressed as follows: ; in, For a moment The predicted values ​​of the final quality indicators for high-level executives, This represents a nonlinear mapping function composed of a fully connected structure, i.e., the high-level prediction task header. For low-level agent task headers, This represents the low-level endpoint quality index prediction result at time t, i.e., the final endpoint quality index prediction result.

[0052] In this embodiment, based on the use of a bidirectional temporal convolutional network to model the temporal characteristics of process variables in complex industrial processes in the quality index prediction model, and considering that the low-level DQN network requires frequent online training and rapid evaluation during the variable time delay optimization process, directly using an independent and complete prediction model would result in high computational overhead, which is not conducive to real-time applications. Therefore, this embodiment introduces a feature sharing mechanism between the high-level prediction task head and the low-level proxy task head to achieve collaborative optimization between the two.

[0053] See the appendix for details. Figure 4Both the high-level prediction task head and the low-level proxy task head employ a shared bidirectional temporal convolutional feature representation during the temporal feature extraction stage. This shared bidirectional temporal convolutional feature representation layer is used to reconstruct multivariate temporal data from the time window. Extract shared temporal features, such as Figure 3 As shown, this is used to characterize the dynamic changes of complex industrial processes. The high-level prediction task head and the low-level proxy task head, following this shared feature layer, are connected to their respective independent output structures to complete prediction and evaluation tasks at different levels.

[0054] To collaboratively train the multi-task feature-sharing network, this embodiment constructs a multi-task joint training loss function based on the independent output structure. The mathematical expression for the joint gradient update of the entire network parameters (i.e., the network parameters of the quality index prediction model and the Q-network parameters of the high-level DQN network and the low-level DQN network) is as follows: ; in, For multi-task joint training loss function, This refers to the high-level prediction error, specifically the mean square error of the final quality index prediction output by the high-level prediction model. This refers to the low-level prediction error, specifically the mean square error of the endpoint quality index prediction output by the low-level surrogate model. To balance the collaborative weighting coefficient of task gradients between high and low layers, this example uses a value of 0.5.

[0055] By introducing the aforementioned feature-sharing mechanism, both the high-level prediction task head and the low-level proxy task head use a unified feature representation, achieving an effective balance between prediction accuracy and training efficiency. This ensures the consistency of evaluation criteria during the optimization of time delay parameters and significantly reduces the time cost of online model training in hierarchical reinforcement learning, thereby improving the method's real-time application capability in complex industrial processes.

[0056] Finally, a working condition fluctuation detection mechanism is constructed. When a significant change in working condition is detected, the variable time lag estimation in steps S2 to S6 is repeated. When no significant change in working condition is detected, the current time lag parameter remains unchanged, and the online prediction results of the endpoint quality index continue to be output.

[0057] In complex and continuous industrial processes, the time lag relationship between multiple variables and the endpoint quality indicators may change at any time depending on the operating conditions.

[0058] For the alumina leaching process, the change in time lag is mainly related to the change in the material transport speed. The slurry flow rate and the caustic soda solution flow rate together determine the material transport status in the production line. In this embodiment, the mixed flow rate formed by these two components is selected as the monitoring quantity to determine whether the current operating condition has experienced a significant fluctuation sufficient to affect the time lag relationship. Let the mixed flow rate at sampling time t be: ; in, This represents the mixed flow rate at sampling time t. This represents the caustic alkali flow rate at sampling time t. This represents the slurry flow rate at sampling time t. Generally, an increase in the mixed flow rate means a faster material transfer velocity in the pipeline, and the corresponding variable time delay tends to decrease. Conversely, a decrease in the mixed flow rate means a decrease in transfer velocity, and the corresponding variable time delay tends to increase. Therefore, online monitoring of changes in the mixed flow rate can provide a criterion for determining whether a new variable time delay estimation is needed. To achieve real-time detection of operating condition fluctuations, this embodiment employs a triggering mechanism constructed using a two-sided cumulative sum statistic. The upper-sided cumulative statistic and the lower-sided cumulative statistic are defined as follows: ; ; in, This represents the reference average value of the mixed flow rate under the current stable operating conditions. This represents the disturbance rejection adjustment, used to reduce the impact of measurement noise and small random fluctuations on the detection results. Characterizes the continuously increasing shift of the mixed flow rate at time t relative to the reference mean. Characterizes the continuously increasing shift of the mixed flow rate relative to the reference mean at time t-1. Characterizes the continuous decreasing shift of the mixed flow rate at time t relative to the reference mean. This characterizes the continuous decreasing shift of the mixed flow rate relative to the reference mean at time t-1. And take: ; When the following equation is satisfied: ; Then it is determined that the current operating condition has changed significantly, where, To preset the trigger threshold, Characterizes the continuously increasing shift of the initial mixed flow rate relative to the reference mean. This characterizes the continuous decreasing shift of the mixed flow rate relative to the reference mean at the initial moment. At this point, a new round of variable time-delay estimation is triggered, involving a re-search and optimization of the time-delay parameters for each process variable. Based on the updated time-delay parameters, the real-time data is re-aligned and the time window is reconstructed. After completing the new round of estimation, the cumulative statistics are reset to zero, and the reference mean is updated according to the new stable operating phase.

[0059] Therefore, it can be seen that the present invention does not re-execute the variable time delay estimation at every sampling moment, but only triggers a new time delay optimization process when a significant fluctuation in the operating condition is detected that may cause the original time delay relationship to fail, thereby taking into account both the time delay tracking accuracy and the efficiency of online application.

[0060] Finally, as Figure 5 Table 1 shows the verification results of the method in this embodiment based on historical operating data from a large alumina plant. Figure 5 As shown in Table 1, under adverse conditions such as measurement noise, operating condition fluctuations, long sampling periods for endpoint quality indicators, and long delays in result feedback in actual industrial processes, the variable time-delay estimation method based on hierarchical reinforcement learning proposed in this invention can adaptively estimate the dynamic time-delay relationship between multiple variables and endpoint quality indicators. Combined with a shared bidirectional temporal convolution prediction model, it achieves accurate prediction of endpoint quality indicators. Experimental results demonstrate that this embodiment exhibits excellent performance in both prediction accuracy and stability, possessing high engineering application value and promotional significance. Table 1, as shown, is an evaluation table of endpoint quality indicator prediction results.

[0061] Table 1. Evaluation of Endpoint Quality Indicator Prediction Results:

[0062] As shown in Table 1, the method of this application achieves high accuracy in product quality testing under unknown working conditions, and has good predictive accuracy and stability for product quality testing.

[0063] Example 4: This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the industrial process quality prediction method based on variable time delay estimation.

[0064] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements an industrial process quality prediction method based on variable time delay estimation as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0065] The processor is used to execute all or part of the steps in the industrial process quality prediction method based on variable time delay estimation as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0066] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the industrial process quality prediction method based on variable time delay estimation described in the above embodiments.

[0067] Example 5: This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0068] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the industrial process quality prediction method based on variable time delay estimation described in various embodiments of this application.

[0069] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) app stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned industrial process quality prediction method based on variable time delay estimation.

[0070] Example 6: This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned industrial process quality prediction method based on variable time delay estimation.

[0071] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

[0073] Those skilled in the art will recognize that the example algorithm steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0074] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for predicting the quality of industrial processes based on variable time delay estimation, characterized in that, include: Obtain real-time process variable sequences in industrial processes; By inputting the real-time process variable sequence into the trained high-level DQN network, the range of the corresponding time delay parameters can be obtained. Input the range of the corresponding time delay parameters into the trained low-level DQN network to obtain the current time delay parameter configuration; Based on the current time delay parameter configuration, perform time alignment and time series sample reconstruction on the real-time process variable sequence; The reconstructed sequence is input into the trained quality index prediction model to obtain the final endpoint quality index prediction result.

2. The industrial process quality prediction method based on variable time delay estimation according to claim 1, characterized in that, Also includes: A training set is constructed, and a high-level DQN network, a low-level DQN network, and a quality indicator prediction model are trained using the training set to obtain the trained high-level DQN network, the trained low-level DQN network, and the trained quality indicator prediction model. The construction of the training set includes: collecting a sequence of multidimensional historical process variables that are causally related to the endpoint quality indicator during industrial operation and the corresponding endpoint quality indicator data; setting a one-to-one time delay parameter for each historical process variable; the time delay parameter is used to characterize the time delay step of the corresponding historical process variable relative to the endpoint quality indicator data.

3. The industrial process quality prediction method based on variable time delay estimation according to claim 2, characterized in that, The process involves training a high-level DQN network, a low-level DQN network, and a quality indicator prediction model using a training set, respectively, to obtain the trained high-level DQN network, the trained low-level DQN network, and the trained quality indicator prediction model, including: The interval range of time delay parameters corresponding to each historical process variable in the training set is coarsely filtered by a high-level DQN network to obtain the interval range of the filtered time delay parameters. By using a low-level DQN network, the time delay parameters of each historical process variable are finely filtered within the range of the filtered time delay parameters to obtain the historical time delay parameter configuration. Input the historical time delay parameters into the low-level agent model to obtain the predicted values ​​of the low-level endpoint quality indicators; Calculate the low-level prediction error between the predicted value of the endpoint quality index and the actual endpoint quality index. Construct a first reinforcement learning reward signal based on the low-level prediction error. Update the network parameters of the low-level DQN network using the first reinforcement learning reward signal to obtain the trained low-level DQN network. The trained low-level DQN network is used to further refine the time delay parameters of each historical process variable within the range of the selected time delay parameters to obtain the updated time delay parameters. Based on the updated time lag parameters, and taking the corresponding time of the endpoint quality index data as the benchmark, the time series of each historical process variable in the training set are differentiated and time-aligned, and reconstructed using a time window of preset length L to obtain the process variable series for quality prediction. By inputting the process variable sequence into the quality indicator prediction model, the predicted values ​​of the final quality indicators at the senior management level are obtained. Calculate the high-level prediction error between the high-level predicted value of the endpoint quality index and the actual endpoint quality index, and use the high-level prediction error to update the quality index prediction model to obtain the trained quality index prediction model. A second reinforcement learning reward signal is constructed based on the high-level prediction error. The network parameters of the high-level DQN network are updated using the second reinforcement learning reward signal to obtain the trained high-level DQN network.

4. The industrial process quality prediction method based on variable time delay estimation according to claim 3, characterized in that, The quality indicator prediction model includes: a forward TCN module, a reverse TCN module, a high-level prediction task head, and a low-level proxy task head. The process variable sequence is input into the forward TCN module to obtain forward time-series features; the process variable sequence is input into the reverse TCN module to obtain reverse time-series features; the forward and reverse time-series features are fused to obtain shared time-series features; the shared time-series features are input into the high-level prediction task head to obtain the high-level endpoint quality indicator prediction value; and the shared time-series features are input into the low-level proxy task head to obtain the low-level endpoint quality indicator prediction result.

5. The industrial process quality prediction method based on variable time delay estimation according to claim 4, characterized in that, The step of inputting the reconstructed sequence into the trained quality index prediction model to obtain the final endpoint quality index prediction result includes: inputting the reconstructed sequence into the trained quality index prediction model, and using the low-level endpoint quality index prediction result as the final endpoint quality index prediction result.

6. The industrial process quality prediction method based on variable time delay estimation according to claim 1, characterized in that, By inputting the real-time process variable sequence into the trained high-level DQN network, the corresponding time delay parameter range is obtained, including: Set a one-to-one time delay parameter for each historical process variable; The real-time process variable sequence is used as the state space of the Markov decision process; The action space of a Markov decision process is represented as low-lag actions, medium-lag actions, and high-lag actions. Establish a mapping relationship between the action space and the time delay parameters; The real-time process variable sequence is input into the trained high-level DQN network to obtain the first Q value of each candidate action. For each process variable in the real-time process variables, select the first target action corresponding to the largest first Q value; For the first target action, the range of the corresponding time delay parameters is obtained based on the mapping relationship between the action space and the time delay.

7. The industrial process quality prediction method based on variable time delay estimation according to claim 1, characterized in that, The step of inputting the corresponding time delay parameter range into the trained low-level DQN network to obtain the current time delay parameter configuration includes: The range of the corresponding time delay parameter is used as the state parameter in the low-level decision-making. Set the fine-tuning step size as the action space in low-level decision-making; Input the interval range of the corresponding time delay parameter into the trained low-level DQN network to obtain the second Q value of each process variable when performing the decrease, maintain, and increase actions respectively; For each process variable, select the second target action corresponding to the largest second Q value; Based on the state parameters in the lower-level decision-making process and the second objective action, the current time delay parameter configuration is obtained.

8. The industrial process quality prediction method based on variable time delay estimation according to claim 1, characterized in that, The step of time alignment and time-series sample reconstruction of the real-time process variable sequence based on the current time delay parameter configuration includes: using the endpoint quality index data as a reference, shifting the i-th process variable according to the current time delay parameter configuration to obtain the reconstructed process variable, as shown in the following expression: ； in, Let i be the reconstructed process variable at time t. Configure the current time delay parameter for the i-th process variable. Let i be the i-th process variable.

9. The industrial process quality prediction method based on variable time delay estimation according to claim 3, characterized in that, A first reinforcement learning reward signal is constructed based on the low-level prediction error, and a second reinforcement learning reward signal is constructed based on the high-level prediction error, including: ； ； in, This is a second reinforcement learning reward signal. As the first reinforcement learning reward signal, This is due to high-level forecasting errors. The low-level prediction error is represented by N, where N is the total number of samples in the training set. Let j be the true endpoint quality index for the j-th sample. Let be the predicted value of the endpoint quality index for the high-level or low-level sample j. H Indicates high-level personnel. L Indicates the lower level.

Images