Multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning

Abstract

The invention relates to an impregnated paper production process multi-parameter collaborative optimization control method based on reinforcement learning, and the method comprises the following steps: S1, building an omnibearing data collection network, achieving the panoramic perception of a production process, and obtaining a real-time data flow; s2, based on a data fusion technology, performing intelligent integration on different sources and different types of data to obtain a fused data set; s3, according to the fused data set and based on a process mechanism, constructing a knowledge graph of the impregnated paper production process; s4, constructing a digital twinborn model of the impregnated paper production line in combination with the fused data set and the knowledge graph; and S5, designing a multi-objective optimization decision framework according to the digital twinborn model, the production plan and the constraint condition, and realizing collaborative optimization of different time scales. According to the invention, multi-parameter collaborative optimization control of the impregnated paper production process is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent manufacturing, and in particular to a multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning. BACKGROUND

[0002] As an important component of surface decoration materials for man-made boards, the production process of impregnated paper involves multiple complex process links such as raw paper pretreatment, impregnating liquid preparation, impregnating drying, hot pressing forming, etc. In the traditional production process of impregnated paper, there is a strong coupling and nonlinear relationship between multiple key parameters such as temperature, pressure, concentration, and speed. Small changes in process parameters may cause significant fluctuations in product quality. The main technical challenges currently faced by the industry include: process parameter optimization relies heavily on experience adjustment, lacking scientific quantitative decision-making basis; multi-parameter collaborative control is difficult, lacking effective coordination mechanism between process links; real-time monitoring and dynamic adjustment capability of production process is insufficient, difficult to adapt to raw material fluctuations and market demand changes; there is a lack of effective means for balancing optimization between energy consumption control and product quality. These technical bottlenecks seriously restrict the stability of impregnated paper product quality and the improvement of production efficiency, and it is urgent to introduce advanced intelligent control technology to break through the limitations of traditional production mode. SUMMARY

[0003] In order to solve the above problems, the purpose of the present application is to provide a multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning, which solves the problem of multi-parameter collaborative optimization control in impregnated paper production process.

[0004] To achieve the above purpose, the present application adopts the following technical scheme:

[0005] The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning comprises the following steps:

[0006] S1: Establish a comprehensive data acquisition network to realize panoramic perception of the production process and obtain real-time data stream;

[0007] S2: Based on data fusion technology, intelligently integrate data of different sources and different types to obtain a fused data set;

[0008] S3: According to the fused data set and based on the process mechanism, a knowledge graph of the impregnated paper production process is constructed;

[0009] S4: Combine the fused data set and the knowledge graph to construct a digital twin model of the impregnated paper production line;

[0010] S5: According to the digital twin model, production plan and constraint conditions, design a multi-objective optimization decision framework to realize collaborative optimization at different time scales.

[0011] Further, a comprehensive data collection network is established to realize panoramic perception of the production process and obtain real-time data flow, as follows:

[0012] A group of sensors is deployed at key process nodes of impregnated paper production, including concentration sensors, temperature sensors, liquid level sensors, and pH value sensors installed in the impregnation tank area; thickness detectors, speed sensors, and tension sensors configured in the coating area; temperature field sensor arrays, humidity sensors, and hot air flow meters arranged in the drying and curing area; and roll diameter measurement and visual sensors set in the winding area.

[0013] For key equipment, including pumps, fans, heaters, and pressure rollers, vibration sensors, current sensors, and bearing temperature sensors are installed to realize real-time monitoring of the health status of the equipment and form an image of the equipment operation status through the collection of equipment operation parameters.

[0014] Near-infrared spectrometers are also deployed to detect paper gum content, and laser scanning is used to detect surface flatness. Temperature and humidity sensors and air pressure sensors are used to monitor the temperature, humidity, and air pressure environment parameters in the workshop.

[0015] A comprehensive data collection network is established to obtain the sensor data mentioned above and collect real-time data flow.

[0016] Further, the comprehensive data collection network adopts a hierarchical data collection network, including a field device layer, a data aggregation layer, and a data center layer, as follows:

[0017] The field device layer connects all sensors to field I / O modules through field buses according to the principle of proximity, and the field bus uses the Profinet protocol.

[0018] The data aggregation layer sets up data aggregation servers in each section, configures industrial computers, runs real-time database software, and connects to the central data center through gigabit Ethernet.

[0019] The data center layer establishes a central database server cluster, uses a time series database design to optimize the storage and query efficiency of time series data, and is equipped with a data backup and recovery system to ensure data security.

[0020] Further, based on data fusion technology, different sources and types of data are intelligently integrated to obtain a fused data set, as follows:

[0021] A unified data preprocessing pipeline was established to perform standardization on data from different sources. First, data cleaning was performed to identify and handle missing, outlier, and duplicate values. Then, data format standardization was implemented, converting all timestamps to standard UTC format, retaining significant digits and standardizing units for numerical data, establishing a standard encoding mapping table for text data, and standardizing resolution and format for image data. Next, time alignment was performed, establishing a time offset compensation mechanism to map all data to a unified time base. Resampling techniques were used for data with different sampling frequencies to ensure consistency of the time series. Finally, data standardization was performed, using Z-score standardization for numerical data and one-hot encoding or label encoding for categorical data to ensure comparability of data with different dimensions and ranges. The result was a preprocessed, standardized multi-source dataset.

[0022] Based on standardized multi-source datasets, and combined with the characteristics of impregnated paper production processes and business needs, a hierarchical data fusion strategy was developed to obtain the fused dataset.

[0023] Furthermore, the hierarchical data fusion strategy includes data-level fusion and feature-level fusion, with each level employing different fusion algorithms and weight allocation mechanisms, as follows: The first layer, data-level fusion, uses a weighted average method. For the immersion tank temperature, the measurements from nine thermocouples are weighted and fused. For equipment vibration data, an adaptive Kalman filter is used, dynamically adjusting the filtering parameters based on the vibration spectrum characteristics to suppress noise interference and improve signal quality. The second layer, feature-level fusion, constructs a comprehensive feature vector, combining the measurements of different physical quantities into a comprehensive index reflecting the system state. Principal component analysis is used for dimensionality reduction, projecting the high-dimensional original features into a low-dimensional space. Independent component analysis is used to separate the mixed signals and extract independent source signal components. Nonlinear relationships between features are identified through mutual information and correlation analysis, and a feature correlation map is constructed. Finally, after two layers of fusion processing, the original multi-source data is transformed into a fused dataset.

[0024] Furthermore, based on the fused dataset and the process mechanism, a knowledge graph of the impregnated paper production process is constructed, as follows:

[0025] Define the semantic structure of domain concepts, establish classification, composition, and dependency relationships between concepts, and form a knowledge framework for the field of impregnated paper production;

[0026] Based on mechanistic modeling and equation integration, the resin penetration into paper fibers during impregnation is a key physical process, following an extended form of Darcy's law. The penetration rate is related to pressure gradient, liquid viscosity, and the permeability of porous media.

[0027] ;

[0028] Where: v is the permeation velocity, k is the effective permeability, and μ is the dynamic viscosity. P is the pressure gradient, ρ is the liquid density, g is the gravitational acceleration, σ is the surface tension, θ is the contact angle, and r is the pore radius;

[0029] In the knowledge graph, the mechanism equation is represented by the MechanisticEquation entity. Each physical quantity in the equation is mapped to the corresponding Parameter entity, and the connection is established through the hasVariable relationship. The calibration results of the equation parameters are stored in the EquationParameter entity, which contains parameter values, calibration accuracy, and applicable range information.

[0030] The drying kinetic equation is:

[0031] ;

[0032] Where X is the wet basis moisture content, t is time, and h is the time. m Here, A is the mass transfer coefficient; A is the heat transfer area, m². s For dry basis quality, Y s Y represents the surface vapor concentration. ∞ Where T is the ambient steam concentration, and k(T,RH) is the drying constant. To achieve equilibrium moisture content;

[0033] The effect of temperature on the drying constant follows the Arrhenius relation:

[0034] ;

[0035] Where: k0 is the pre-exponential factor, E a R is the activation energy, R is the gas constant, and T is the absolute temperature;

[0036] The curing process of thermosetting resins follows the laws of chemical reaction kinetics, and the change in degree of cure over time can be described by the Kamal model:

[0037] ;

[0038] Where α is the degree of curing, k1 and k2 are the reaction rate constants, and m and n are the reaction orders;

[0039] A mechanistic knowledge network is formed in the knowledge graph, which is connected to the relevant process through the hasEquation relationship, connected to the influencing process parameters through the effectsParameter relationship, and connected to the physicochemical principle through the basedOnPrinciple relationship.

[0040] Expert experience is converted into standard fuzzy reasoning rules, using the Mamdani reasoning method;

[0041] Based on the knowledge framework, mechanism knowledge modeling and equation integration, and fuzzy reasoning rules in the field of impregnated paper production, a knowledge graph of the impregnated paper production process is constructed.

[0042] Furthermore, by combining the fused dataset and knowledge graph, a digital twin model of the impregnated paper production line is constructed, as follows:

[0043] Based on the fused dataset and knowledge graph, the digital twin model of the impregnated paper production line adopts a five-dimensional architecture, including physical space, virtual space, connection layer, data layer, and service layer. The physical space includes the actual production equipment, sensor network, control system, and material flow. The virtual space contains the digital representation of the physical system, integrating geometric model, physical model, behavioral model, and knowledge model. The connection layer enables real-time bidirectional interaction between the physical and virtual spaces, including data acquisition, status synchronization, and control command issuance. The data layer manages structured knowledge from the fused dataset and knowledge graph. The service layer provides various intelligent services based on the digital twin model, including status monitoring, fault diagnosis, and performance prediction.

[0044] The digital twin model has two operating modes: real-time synchronization and predictive simulation. In real-time synchronization mode, the digital twin model keeps time synchronized with the physical system, receives sensor data in real time, updates the state of the digital twin model, and achieves accurate mapping of the physical system. In predictive simulation mode, forward simulation is performed based on the current state and boundary conditions to predict the future evolution trajectory of the system.

[0045] Furthermore, virtual space includes a digital representation of the physical system, integrating geometric models, physical models, behavioral models, and knowledge models. Specifically:

[0046] The geometric model establishes a three-dimensional digital representation of the production line, including equipment geometric models, plant space models, and material flow path models. Accurate geometric information is obtained by importing CAD data. The physical model integrates the mechanistic equations in the knowledge graph to establish a mathematical model describing the physical phenomena.

[0047] The behavioral model describes the dynamic response characteristics of the system, including control system behavior, equipment motion behavior, and material flow behavior; the control system behavior adopts a state-space model.

[0048] ;

[0049] Where x is the state vector, u is the input vector, y is the output vector, w and v are the process noise and measurement noise, and A, B, C, and D are the system matrices; The time derivative of the state vector;

[0050] The dynamic behavior of the equipment is governed by multibody dynamics equations:

[0051] ;

[0052] Where q is the generalized coordinate, M is the mass matrix, C is the damping matrix, K is the stiffness matrix, and Q is the generalized force. c For binding force; It is a generalized acceleration vector; The vector is the generalized velocity vector; t represents time t.

[0053] The knowledge model employs a hybrid modeling approach, combining mechanistic and data-driven models:

[0054] ;

[0055] Among them, f physics For the physical mechanism model, f ML This is a machine learning model, where α is the fusion weight, dynamically adjusted using an adaptive algorithm. This represents the predicted output value of the knowledge model. This is the physical model parameter vector; For machine learning model parameter vectors;

[0056] ;

[0057] Among them, e physics and e ML The historical errors of the physical model and the machine learning model are respectively, σ is the activation function, and w α Here, x(t) is the weight parameter vector; x(t) is the input feature at the current time step; e physics (t) represents the historical prediction error of the physical model at time t; e ML (t) represents the historical prediction error of the machine learning model at time t; b α This is the bias parameter, used to adjust the baseline value of the weights.

[0058] Furthermore, in predictive simulation mode, forward simulation is performed based on the current state and boundary conditions to predict the future evolution trajectory of the system, as follows:

[0059] Short-term prediction is based on time series analysis and recurrent neural networks, and a long short-term memory network is used to capture temporal dependencies.

[0060] ;

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] in, The hidden state at time step t; Forgotten Gate; For input gates; These are candidate values, i.e., candidate memory cells at the current moment; The cell state represents the current state of the memory cell. For output gate; The current hidden state; , , , These are the weight matrices for the forget gate, input gate, candidate values, and output gate, respectively. , , , σ represents the bias vectors for the forget gate, input gate, candidate value, and output gate, respectively; σ is the sigmoid function; ⊙ is the Hadamard product; X t The input represents the time step t;

[0067] Using a state-space model for medium-term forecasting:

[0068] ;

[0069] Where, x t Let x be the state vector at time t; t+1 Let u be the state vector at the next time step (t+1); t Let y be the control input vector at time t; t Let A be the observed output vector at time t; B be the state transition matrix; C be the control input matrix; D be the output matrix; and W be the feedforward matrix. t v is the process noise vector; t For the observed noise vector;

[0070] The parameter matrix is ​​obtained through system identification:

[0071] ;

[0072] Where θ is the parameter vector to be estimated; y represents the optimal estimate of the parameter; N is the total length of the observed data; t The actual observed value at time t; These are model predictions based on parameter θ; λ represents the square norm of the prediction error; λ is the regularization coefficient.

[0073] Furthermore, a multi-objective optimization decision-making framework is used to achieve collaborative optimization at different time scales, as detailed below:

[0074] Production process optimization involves multiple conflicting objectives such as quality, cost, efficiency, and energy consumption. A multi-objective optimization method is used to find the Pareto optimal solution set.

[0075] minF(x)=[f1(x),f2(x),f3(x),f4(x)] T ;

[0076] subject to: g(x)≤0,h(x)=0;

[0077] Where f1(x) is the quality index, f2(x) is the production cost, f3(x) is the energy consumption index, f4(x) is the production efficiency; g(x) represents the maximum load of the equipment; and h(x) represents the production quota.

[0078] Reinforcement learning adaptive control employs a deep deterministic policy gradient algorithm to learn the adaptive control policy, where the Actor network is updated:

[0079] ;

[0080] in, This represents the parameter vector of the Actor network; For the deterministic policy function; s t Let be the state at time t; 'a' be the action variable. The state distribution represents the behavioral strategy; β represents the behavioral strategy. For Critic network; s represents the state; This represents the gradient of the Q-function with respect to the action. These are the parameters of the Critic network; This represents the integration over the state under the behavioral policy distribution; J is the objective function;

[0081] Critic Network Update:

[0082] ;

[0083] Among them, y t TD target value; For experience replay buffer; r t For the immediate reward obtained; s t+1 The next state;

[0084] Target network soft update:

[0085] ;

[0086] in, Target Critic network parameters; Here are the target Actor network parameters; τ is the soft update coefficient.

[0087] Based on the predictive capabilities of digital twin models, rolling optimization is achieved using nonlinear model predictive control:

[0088] ;

[0089] ;

[0090] Where L(x,u) is the stage cost function, V f (x) is the terminal cost function, N is the prediction time domain length; k is the time step index within the prediction time domain; f(x) k ,u k ) is the system state transition function; To control the set of input constraints; For the set of state constraints; For the terminal constraint set; x k ,u k Let these represent the system state vector and control input vector at step k, respectively;

[0091] Cost function Designed as a weighted multi-objective form:

[0092] ;

[0093] Where w1, w2, w3 are weight coefficients, Q, R, S are weight matrices, Δu is the control increment, and u ref To control the reference value vector, x ref This is a vector of state reference values.

[0094] The present invention has the following beneficial effects:

[0095] 1. This invention constructs a comprehensive data acquisition network combined with multi-source heterogeneous data fusion technology, breaking through the limitations of traditional single data sources. It realizes data transparency and standardization throughout the entire process from raw material input to finished product output, significantly improving data quality and information integrity. Secondly, based on the process mechanism, a knowledge graph is constructed, organically combining empirical knowledge, theoretical mechanisms, and real-time data to form a structured knowledge representation system. Furthermore, through the deep integration of digital twin models and reinforcement learning algorithms, real-time synchronous mapping between physical and virtual systems is achieved, providing a high-fidelity simulation verification environment for multi-parameter collaborative optimization.

[0096] 2. This invention, through its objective optimization decision framework, can simultaneously handle multiple constraints such as product quality, production efficiency, energy consumption cost, and equipment safety, thus solving the problem of insufficient adaptability of traditional single-objective optimization methods in complex industrial environments. Attached Figure Description

[0097] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0098] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0099] refer to Figure 1 In this embodiment, a multi-parameter collaborative optimization control method for the impregnated paper production process based on reinforcement learning is provided, including the following steps:

[0100] S1: Establish a comprehensive data acquisition network to achieve panoramic perception of the production process and acquire real-time data streams;

[0101] S2: Based on data fusion technology, data from different sources and of different types are intelligently integrated to obtain a fused dataset;

[0102] S3: Based on the fused dataset and the process mechanism, construct a knowledge graph of the impregnated paper production process;

[0103] S4: Combine the fused dataset and knowledge graph to construct a digital twin model of the impregnated paper production line;

[0104] S5: Based on the digital twin model, production plan, and constraints, design a multi-objective optimization decision framework to achieve collaborative optimization at different time scales.

[0105] In this embodiment, a comprehensive data acquisition network is established to achieve panoramic perception of the production process and acquire real-time data streams, as detailed below:

[0106] Sensor clusters are deployed at key process nodes in the production of impregnated paper. Concentration sensors (conductivity type, refractive type), temperature sensors (thermocouple, RTD), liquid level sensors (ultrasonic, pressure type), and pH sensors are installed in the impregnation tank area; thickness detectors (laser, eddy current), speed sensors (encoder, radar), and tension sensors (pressure type) are configured in the coating area; temperature field sensor arrays, humidity sensors, and hot air flow meters are deployed in the drying and curing area; and roll diameter measurement and vision sensors are set up in the winding area.

[0107] For key equipment, including pumps, fans, heaters, and pressure rollers, vibration sensors, current sensors, and bearing temperature sensors are installed to achieve real-time monitoring of the equipment's health status. By collecting equipment operating parameters, a profile of the equipment's operating status is formed.

[0108] Near-infrared spectrometers were deployed to detect the glue content of paper, laser scanning was set to detect surface flatness, and temperature and humidity sensors and air pressure sensors were used to monitor the temperature, humidity and air pressure environmental parameters of the workshop.

[0109] Construct a comprehensive data acquisition network to obtain data from the aforementioned sensors and collect real-time data streams.

[0110] In this embodiment, the comprehensive data acquisition network adopts a layered data acquisition network, including a field device layer, a data aggregation layer, and a data center layer, as detailed below:

[0111] At the field device layer, all sensors are connected to the field I / O modules via a fieldbus according to proximity. A modular design is adopted, supporting multiple signal types (analog input AI, digital input DI, pulse input PI, etc.). The fieldbus uses the Profinet protocol, with a transmission speed of 100Mbps, and supports hot-swapping and redundant configuration.

[0112] Data aggregation layer: Data aggregation servers are set up in each work section, configured with industrial-grade computers (Intel i7 processor, 16GB memory, 1TB SSD storage), running real-time database software, and connected to the central data center via gigabit Ethernet.

[0113] Data Center Layer: A central database server cluster is established. The database adopts a time-series database design to optimize the storage and query efficiency of time-series data, and is equipped with a data backup and recovery system to ensure data security.

[0114] In this embodiment, based on data fusion technology, data from different sources and of different types are intelligently integrated to obtain a fused dataset, as follows:

[0115] A unified data preprocessing pipeline is established to perform standardized processing on data from different sources. First, data cleaning is performed to identify and handle missing values ​​(using strategies such as forward imputation, backward imputation, linear interpolation, and mean imputation, selecting the appropriate method based on data characteristics), outliers (detected using the 3σ criterion, quartile method, and isolated forest algorithm; data exceeding the normal range is marked or corrected), and duplicate values ​​(deduplicated based on timestamps and data source identifiers). Then, data format standardization is performed, converting all timestamps to standard UTC format, retaining significant digits for numerical data, and standardizing unit systems (temperature to degrees Celsius, pressure to MPa, velocity to...). First, a standard encoding mapping table is established for text data (e.g., m / min), and image data is standardized in resolution and format. Next, time alignment processing is performed, and a time offset compensation mechanism is established to map all data to a unified time base. For data with different sampling frequencies, resampling techniques are used (interpolation for upsampling and filtering to prevent aliasing for downsampling) to ensure the consistency of the time series. Finally, data standardization is performed: Z-score standardization (mean 0, standard deviation 1) is used for numerical data, and one-hot encoding or label encoding is used for categorical data to ensure comparability of data with different dimensions and ranges. This results in a preprocessed, standardized multi-source dataset.

[0116] Based on standardized multi-source datasets, and combined with the characteristics of impregnated paper production processes and business needs, a hierarchical data fusion strategy was developed to obtain the fused dataset.

[0117] In this embodiment, the hierarchical data fusion strategy includes data-level fusion (fusion of redundant measurements from similar sensors) and feature-level fusion (feature combination of different types of data). Each level employs different fusion algorithms and weight allocation mechanisms, as follows: The first layer, data-level fusion, uses a weighted average method. For the immersion tank temperature, the measurements from nine thermocouples are weighted and fused. For equipment vibration data, an adaptive Kalman filter is used, dynamically adjusting the filtering parameters based on the vibration spectrum characteristics to suppress noise interference and improve signal quality. The second layer, feature-level fusion, constructs a comprehensive feature vector, combining the measurements of different physical quantities into a comprehensive index reflecting the system state. Principal component analysis (PCA) is used for dimensionality reduction, projecting the high-dimensional original features into a low-dimensional space while retaining more than 95% of the information. Independent component analysis (ICA) is used to separate the mixed signals and extract independent source signal components. Mutual information and correlation analysis are used to identify nonlinear relationships between features and construct a feature correlation map. Finally, after two layers of fusion processing, the original multi-source data is transformed into a fused dataset.

[0118] In this embodiment, a knowledge graph of the impregnated paper production process is constructed based on the fused dataset and the process mechanism, as follows:

[0119] Define the semantic structure of domain concepts, establish classification, composition, and dependency relationships between concepts, and form a knowledge framework for the field of impregnated paper production;

[0120] The `ProcessStep` class is defined as an abstract concept of a process step, specifically including subclasses `Preparation`, `Impregnation`, `Coating`, `Drying`, `Curing`, and `Winding`. Each process step is connected to relevant process parameters through the `hasParameter` relationship, to the equipment used through the `usesEquipment` relationship, and to the input and output materials through the `hasInput` and `hasOutput` relationships. Process steps are sequentially dependent on each other through the `precedes` and `follows` relationships, forming a complete process flow chain. A time ontology is introduced to model the dynamic characteristics of the process, including time attributes such as duration, start time, and end time.

[0121] Equipment is a crucial component of the production process. Classified by function, it includes subcategories such as ImpregnationTank, CoatingHead, DryingChamber, Pump, and Sensor. Equipment is described by its structural composition through the `hasComponent` relation, its spatial location through the `locatedIn` relation, and its operational status through the `hasState` relation. Materials include subcategories such as BasePaper, Resin, Solvent, and Additive. Each material possesses specific physicochemical properties, connected by the `hasProperty` relation, linking related property parameters. Materials exhibit mixing, dissolution, and reaction chemical relationships, which are essential components of process mechanism knowledge.

[0122] Process parameters and quality indicators are key numerical concepts in the knowledge graph, and their numerical characteristics and constraints are modeled. Parameters include subclasses such as Temperature, Pressure, Concentration, Viscosity, Speed, and Tension. Quality indicators include subclasses such as ResinContent, Thickness, UniformityIndex, and SurfaceQuality. Influence relationships between parameters and indicators are established through the `affects` relationship, and correlation relationships are established through the `correlatedWith` relationship. The concept of constraint is introduced to describe the constraints on parameter values, including minimum value constraint (minValue), maximum value constraint (maxValue), and target value constraint (targetValue).

[0123] Modeling Causal and Influence Relationships: Causal relationships are the most important relationship type in knowledge graphs, used to model the impact mechanism of parameter changes on quality outcomes. The `causes` relation is defined to represent direct causal relationships, including attributes such as influence strength, time delay, and nonlinearity. The `affects` relation is used to represent relatively broad influence relationships, without emphasizing strict causality. The directionality of the relationship is naturally expressed through directed edges in RDF, while positive and negative effects are introduced to refine the properties of the influence. For complex multi-factor influence relationships, the `complexAffects` relation is defined, associated with an influence function (`InfluenceFunction`), which contains specific mathematical expressions and parameter settings. Time delay is represented by the `hasDelay` data attribute, supporting the modeling of lag effects and dynamic response characteristics.

[0124] Hierarchical relationships are used to express classification and inclusion relationships between concepts, implemented using standard OWL class hierarchies and object attribute hierarchies. In addition to the simple is-a relationship, part-of relationships are defined to represent composition, component-of relationships to represent parts, and belong-to relationships to represent subordination. These relationships support hierarchical organization of knowledge and multi-level reasoning. Spatial relationships are expressed through locatedIn, adjacentTo, upstream, and downstream relationships, supporting spatial reasoning and propagation analysis. Temporal relationships are expressed through before, after, during, and overlaps relationships, supporting temporal reasoning and event association analysis. Quantitative relationships are expressed through data attributes such as hasValue, hasThreshold, and hasTarget, connecting conceptual entities with specific numerical data.

[0125] The dynamic characteristics of the production process need to be modeled through dynamic relationships to support the expression of state changes and process evolution. The `StateTransition` class is defined to represent state transitions, connecting the states before and after the transition through `fromState` and `toState` relationships, and connecting the conditions or events that trigger the transition through `triggeredBy` relationships. The concept of an event is introduced as a driving factor for state changes, with attributes such as occurrence time, duration, and severity. Process evolution is modeled using the `ProcessEvolution` class, describing the trend of parameter changes over time, including information such as trend type, change rate, and prediction interval. A versioned relationship management mechanism is established, using `validFrom` and `validTo` timestamps to mark the validity period of relationships, supporting dynamic knowledge updates and historical traceability.

[0126] Based on mechanistic modeling and equation integration, the resin penetration into paper fibers during impregnation is a key physical process, following an extended form of Darcy's law. The penetration rate is related to pressure gradient, liquid viscosity, and the permeability of porous media.

[0127] ;

[0128] Where: v is the permeation velocity, k is the effective permeability, and μ is the dynamic viscosity. P is the pressure gradient, ρ is the liquid density, g is the gravitational acceleration, σ is the surface tension, θ is the contact angle, and r is the pore radius;

[0129] In the knowledge graph, the mechanism equation is represented by the MechanisticEquation entity. Each physical quantity in the equation is mapped to the corresponding Parameter entity, and the connection is established through the hasVariable relationship. The calibration results of the equation parameters are stored in the EquationParameter entity, which contains parameter values, calibration accuracy, and applicable range information.

[0130] The drying kinetic equation is:

[0131] ;

[0132] Where X is the wet basis moisture content, t is time, and h is the time. m Here, A is the mass transfer coefficient; A is the heat transfer area, m². s For dry basis quality, Y s Y represents the surface vapor concentration. ∞ Where T is the ambient vapor concentration, and k(T,RH) is the drying constant (a function of temperature and relative humidity). To achieve equilibrium moisture content;

[0133] The effect of temperature on the drying constant follows the Arrhenius relation:

[0134] ;

[0135] Where: k0 is the pre-exponential factor, E a R is the activation energy, R is the gas constant, and T is the absolute temperature;

[0136] The curing process of thermosetting resins follows the laws of chemical reaction kinetics, and the change in degree of cure over time can be described by the Kamal model:

[0137] ;

[0138] Where α is the degree of curing, k1 and k2 are the reaction rate constants, and m and n are the reaction orders;

[0139] A mechanistic knowledge network is formed in the knowledge graph, which is connected to the relevant process through the hasEquation relationship, connected to the influencing process parameters through the effectsParameter relationship, and connected to the physicochemical principle through the basedOnPrinciple relationship.

[0140] Expert experience is converted into standard fuzzy reasoning rules, using the Mamdani reasoning method;

[0141] Typical quality control rules are as follows:

[0142] Rule R1: ; Where concentration is the concentration; temperature is the temperature; speed is the speed; and resin content uniformity is the uniformity of resin content. Rule R2: ; Where viscosity is the viscosity; pressure is the pressure; and penetration effect is the penetration effect. Rule R3: ; Where, drying temperature is the drying temperature; humidity is the humidity; foaming risk is the foaming risk; The trigger strength of each rule is determined by... Norm calculation: ; in, The matching degree (activation degree, membership degree) of rule i is the activation strength of this fuzzy rule under the current input. Representing diversity Norm operations integrate the membership degrees of each input variable to obtain the overall matching degree; This represents the current value x of the nth input variable. n For fuzzy set A n The degree of membership.

[0143] Based on the knowledge framework, mechanism knowledge modeling and equation integration, and fuzzy reasoning rules in the field of impregnated paper production, a knowledge graph of the impregnated paper production process is constructed.

[0144] In this embodiment, a digital twin model of the impregnated paper production line is constructed by combining the fused dataset and knowledge graph, as detailed below:

[0145] Based on the fused dataset and knowledge graph, the digital twin model of the impregnated paper production line adopts a five-dimensional architecture, including physical space, virtual space, connection layer, data layer, and service layer. The physical space includes the actual production equipment, sensor network, control system, and material flow. The virtual space contains the digital representation of the physical system, integrating geometric model, physical model, behavioral model, and knowledge model. The connection layer enables real-time bidirectional interaction between the physical and virtual spaces, including data acquisition, status synchronization, and control command issuance. The data layer manages structured knowledge from the fused dataset and knowledge graph. The service layer provides various intelligent services based on the digital twin model, including status monitoring, fault diagnosis, and performance prediction.

[0146] The digital twin model operates in two modes: real-time synchronization and predictive simulation. In real-time synchronization mode, the digital twin model maintains time synchronization with the physical system, receiving sensor data in real time to update its state and achieve accurate mapping of the physical system. In predictive simulation mode, forward simulation is performed based on the current state and boundary conditions to predict the future evolution trajectory of the system, providing forward-looking support for decision-making. The fusion of these two modes is achieved through a parallel computing architecture. The real-time synchronization thread ensures the current accuracy of the model, while the predictive simulation thread provides future insights. A mode switching mechanism is established to dynamically select the appropriate operating mode based on application requirements and computing resources.

[0147] In this embodiment, the virtual space includes a digital representation of the physical system, integrating geometric models, physical models, behavioral models, and knowledge models. Specifically:

[0148] The geometric model establishes a three-dimensional digital representation of the production line, including equipment geometric models, plant space models, and material flow path models. Accurate geometric information is obtained by importing CAD data. The physical model integrates the mechanistic equations in the knowledge graph to establish a mathematical model describing the physical phenomena.

[0149] The behavioral model describes the dynamic response characteristics of the system, including control system behavior, equipment motion behavior, and material flow behavior; the control system behavior adopts a state-space model.

[0150] ;

[0151] Where x is the state vector, u is the input vector, y is the output vector, w and v are the process noise and measurement noise, and A, B, C, and D are the system matrices; The time derivative of the state vector;

[0152] The dynamic behavior of the equipment is governed by multibody dynamics equations:

[0153] ;

[0154] Where q is the generalized coordinate, M is the mass matrix, C is the damping matrix, K is the stiffness matrix, and Q is the generalized force. c For binding force; It is a generalized acceleration vector; The vector is the generalized velocity vector; t represents time t.

[0155] The knowledge model employs a hybrid modeling approach, combining mechanistic and data-driven models:

[0156] ;

[0157] Among them, fphysics For the physical mechanism model, f ML This is a machine learning model, where α is the fusion weight, dynamically adjusted using an adaptive algorithm. This represents the predicted output value of the knowledge model. This is the physical model parameter vector; For machine learning model parameter vectors;

[0158] ;

[0159] Among them, e physics and e ML The historical errors of the physical model and the machine learning model are respectively, σ is the activation function, and w α The vector represents the weight parameters; x(t) represents the input features (process parameters, etc.) at the current time; e physics (t) represents the historical prediction error of the physical model at time t; e ML (t) represents the historical prediction error of the machine learning model at time t; b α This is the bias parameter, used to adjust the baseline value of the weights.

[0160] In this embodiment, in predictive simulation mode, forward simulation is performed based on the current state and boundary conditions to predict the future evolution trajectory of the system, as follows:

[0161] Short-term prediction is based on time series analysis and recurrent neural networks, and a long short-term memory network is used to capture temporal dependencies.

[0162] ;

[0163] ;

[0164] ;

[0165] ;

[0166] ;

[0167] ;

[0168] in, The hidden state at time step t; Forgotten Gate; For input gates; These are candidate values, i.e., candidate memory cells at the current moment; The cell state represents the current state of the memory cell. For output gate; The current hidden state; , , , These are the weight matrices for the forget gate, input gate, candidate values, and output gate, respectively. , , , σ represents the bias vectors for the forget gate, input gate, candidate value, and output gate, respectively; σ is the sigmoid function; ⊙ is the Hadamard product; X t The input represents the time step t;

[0169] Using a state-space model for medium-term forecasting:

[0170] ;

[0171] Where, x t Let x be the state vector at time t; t+1 Let u be the state vector at the next time step (t+1); t Let y be the control input vector at time t; t Let A be the observed output vector at time t; B be the state transition matrix; C be the control input matrix; D be the output matrix; and W be the feedforward matrix. t v is the process noise vector; t For the observed noise vector;

[0172] The parameter matrix is ​​obtained through system identification:

[0173] ;

[0174] Where θ is the parameter vector to be estimated; y represents the optimal estimate of the parameter; N is the total length of the observed data; t The actual observed value at time t; These are model predictions based on parameter θ; λ represents the square norm of the prediction error; λ is the regularization coefficient.

[0175] In this embodiment, a multi-objective optimization decision framework is used to achieve collaborative optimization at different time scales, as detailed below:

[0176] Production process optimization involves multiple conflicting objectives such as quality, cost, efficiency, and energy consumption. A multi-objective optimization method is used to find the Pareto optimal solution set.

[0177] minF(x)=[f1(x),f2(x),f3(x),f4(x)] T ;

[0178] subject to: g(x)≤0,h(x)=0;

[0179] Where f1(x) is the quality index (maximization), f2(x) is the production cost (minimization), f3(x) is the energy consumption index (minimization), and f4(x) is the production efficiency (maximization); g(x) represents the maximum load of the equipment; and h(x) represents the production quota.

[0180] Reinforcement learning adaptive control employs a deep deterministic policy gradient algorithm to learn the adaptive control policy, where the Actor network is updated:

[0181] ;

[0182] in, This represents the parameter vector of the Actor network; For the deterministic policy function; s t Let be the state at time t; 'a' be the action variable. The state distribution represents the behavioral strategy; β represents the behavioral strategy. For Critic network; s represents the state; This represents the gradient of the Q-function with respect to the action. These are the parameters of the Critic network; This represents the integration over the state under the behavioral policy distribution; J is the objective function;

[0183] Critic Network Update:

[0184] ;

[0185] Among them, y t TD target value; For experience replay buffer; r t For the immediate reward obtained; s t+1 The next state;

[0186] Target network soft update:

[0187] ;

[0188] in, Target Critic network parameters; Here are the target Actor network parameters; τ is the soft update coefficient.

[0189] Based on the predictive capabilities of digital twin models, rolling optimization is achieved using nonlinear model predictive control:

[0190] ;

[0191] ;

[0192] Where L(x,u) is the stage cost function, Vf (x) is the terminal cost function, N is the prediction time domain length; k is the time step index within the prediction time domain; f(x) k ,u k ) is the system state transition function; To control the set of input constraints; For the set of state constraints; For the terminal constraint set; x k ,u k Let these represent the system state vector and control input vector at step k, respectively;

[0193] Cost function Designed as a weighted multi-objective form:

[0194] ;

[0195] Where w1, w2, w3 are weight coefficients, Q, R, S are weight matrices, Δu is the control increment, and u ref To control the reference value vector, x ref This is a vector of state reference values.

[0196] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0197] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0198] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a processFigure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0200] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning, characterized in that, Includes the following steps: S1: Establish a comprehensive data acquisition network to achieve panoramic perception of the production process and acquire real-time data streams; S2: Based on data fusion technology, data from different sources and of different types are intelligently integrated to obtain a fused dataset; S3: Based on the fused dataset and the process mechanism, construct a knowledge graph of the impregnated paper production process; S4: Combine the fused dataset and knowledge graph to construct a digital twin model of the impregnated paper production line; S5: Based on the digital twin model, production plan, and constraints, design a multi-objective optimization decision framework to achieve collaborative optimization at different time scales.

2. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 1, characterized in that, The establishment of a comprehensive data acquisition network to achieve panoramic perception of the production process and acquire real-time data streams is detailed below: Sensor clusters are deployed at process nodes in the impregnated paper production process. Concentration sensors, temperature sensors, liquid level sensors, and pH sensors are installed in the impregnation tank area; thickness detectors, speed sensors, and tension sensors are configured in the coating area; temperature field sensor arrays, humidity sensors, and hot air flow meters are deployed in the drying and curing area; and roll diameter measurement and vision sensors are set up in the winding area. For key equipment, including pumps, fans, heaters, and pressure rollers, vibration sensors, current sensors, and bearing temperature sensors are installed to achieve real-time monitoring of the equipment's health status. By collecting equipment operating parameters, a profile of the equipment's operating status is formed. Near-infrared spectrometers were deployed to detect the glue content of paper, laser scanning was set to detect surface flatness, and temperature and humidity sensors and air pressure sensors were used to monitor the temperature, humidity and air pressure environmental parameters of the workshop. A data acquisition network is constructed to obtain the aforementioned sensor data and collect real-time data streams.

3. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 2, characterized in that, The data acquisition network adopts a layered data acquisition network, including a field device layer, a data aggregation layer, and a data center layer, as detailed below: At the field device layer, all sensors are connected to the field I / O modules via a fieldbus according to the principle of proximity. The fieldbus adopts the Profinet protocol. Data aggregation layer: Data aggregation servers are set up in each work section and connected to the central data center via gigabit Ethernet; Data Center Layer: A central database server cluster is established. The database adopts a time-series database design to optimize the storage and query efficiency of time-series data, and is equipped with a data backup and recovery system to ensure data security.

4. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 1, characterized in that, The aforementioned data fusion technology intelligently integrates data from different sources and of different types to obtain a fused dataset. Specifically: A unified data preprocessing pipeline is established to perform standardization processing on data from different sources. First, data cleaning is performed to identify and handle missing, outlier, and duplicate values. Then, data format standardization is implemented, converting all timestamps to standard UTC format, retaining significant digits and standardizing units for numerical data, establishing a standard encoding mapping table for text data, and standardizing resolution and format for image data. Next, time alignment processing is performed, establishing a time offset compensation mechanism to map all data to a unified time base, and using resampling technology for data with different sampling frequencies to ensure the consistency of the time series. Finally, data standardization is performed, using Z-score standardization for numerical data and one-hot encoding or label encoding for categorical data to ensure comparability of data with different dimensions and ranges. This results in a preprocessed, standardized multi-source dataset. Based on this standardized multi-source dataset, and considering the characteristics of impregnated paper production processes and business needs, a hierarchical data fusion strategy is formulated to obtain the fused dataset.

5. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 4, characterized in that, The hierarchical data fusion strategy includes data-level fusion and feature-level fusion. Each level employs different fusion algorithms and weight allocation mechanisms, as follows: The first layer, data-level fusion, uses a weighted average method. For the immersion tank temperature, the measurements from nine thermocouples are weighted and fused. For equipment vibration data, an adaptive Kalman filter is used, dynamically adjusting the filter parameters based on the vibration spectrum characteristics to suppress noise interference and improve signal quality. The second layer, feature-level fusion, constructs a comprehensive feature vector, combining the measurements of different physical quantities into a comprehensive index reflecting the system state. Principal component analysis is used for dimensionality reduction, projecting the high-dimensional original features into a low-dimensional space. Independent component analysis is used to separate the mixed signals and extract independent source signal components. Nonlinear relationships between features are identified through mutual information and correlation analysis, and a feature correlation map is constructed. Finally, after two layers of fusion processing, the original multi-source data is transformed into a fused dataset.

6. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 1, characterized in that, Based on the fused dataset and the process mechanism, a knowledge graph of the impregnated paper production process is constructed, as follows: Define the semantic structure of domain concepts, establish classification, composition, and dependency relationships between concepts, and form a knowledge framework for the field of impregnated paper production; Based on mechanistic modeling and equation integration, the penetration of resin into paper fibers during impregnation is a key physical process, following an extended form of Darcy's law. The penetration rate is related to pressure gradient, liquid viscosity, and the permeability of porous media. ； Where: v is the permeation velocity, k is the effective permeability, and μ is the dynamic viscosity. P is the pressure gradient, ρ is the liquid density, g is the gravitational acceleration, σ is the surface tension, θ is the contact angle, and r is the pore radius; In the knowledge graph, the mechanism equation is represented by the MechanisticEquation entity. Each physical quantity in the equation is mapped to the corresponding Parameter entity, and the connection is established through the hasVariable relationship. The calibration results of the equation parameters are stored in the EquationParameter entity, which contains parameter values, calibration accuracy, and applicable range information. The drying kinetic equation is: ； Where X is the wet basis moisture content, t is time, and h is the time. m Here, A is the mass transfer coefficient; A is the heat transfer area, m². s For dry basis quality, Y s Y represents the surface vapor concentration. ∞ Where T is the ambient steam concentration, and k(T,RH) is the drying constant. To achieve equilibrium moisture content; The effect of temperature on the drying constant follows the Arrhenius relation: ； Where: k0 is the pre-exponential factor, E a R is the activation energy, R is the gas constant, and T is the absolute temperature; The curing process of thermosetting resins follows the laws of chemical reaction kinetics, and the change in degree of cure over time can be described by the Kamal model: ； Where α is the degree of curing, k1 and k2 are the reaction rate constants, and m and n are the reaction orders; A mechanistic knowledge network is formed in the knowledge graph, which is connected to the relevant process through the hasEquation relationship, connected to the influencing process parameters through the effectsParameter relationship, and connected to the physicochemical principle through the basedOnPrinciple relationship. Expert experience is converted into standard fuzzy reasoning rules, using the Mamdani reasoning method; Based on the knowledge framework, mechanism knowledge modeling and equation integration, and fuzzy reasoning rules in the field of impregnated paper production, a knowledge graph of the impregnated paper production process is constructed.

7. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 1, characterized in that, The digital twin model of the impregnated paper production line is constructed by combining the fused dataset and knowledge graph, as follows: Based on the fused dataset and knowledge graph, the digital twin model of the impregnated paper production line adopts a five-dimensional architecture design, including physical space, virtual space, connection layer, data layer, and service layer; the physical space includes the actual production equipment, sensor network, control system, and material flow; the virtual space includes the digital representation of the physical system, integrating geometric model, physical model, behavioral model, and knowledge model; the connection layer realizes real-time bidirectional interaction between the physical space and the virtual space, including data acquisition, status synchronization, and control command issuance functions; The data layer manages structured knowledge from the fused dataset and knowledge graph; the service layer provides various intelligent services based on the digital twin model, including status monitoring, fault diagnosis, and performance prediction. The digital twin model has two operating modes: real-time synchronization and predictive simulation. In real-time synchronization mode, the digital twin model keeps time synchronized with the physical system, receives sensor data in real time, updates the state of the digital twin model, and achieves accurate mapping of the physical system. In predictive simulation mode, forward simulation is performed based on the current state and boundary conditions to predict the future evolution trajectory of the system.

8. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 7, characterized in that, The virtual space contains a digital representation of the physical system, integrating geometric models, physical models, behavioral models, and knowledge models. Specifically: The geometric model establishes a three-dimensional digital representation of the production line, including equipment geometric models, plant space models, and material flow path models. Accurate geometric information is obtained by importing CAD data. The physical model integrates the mechanistic equations in the knowledge graph to establish a mathematical model describing the physical phenomena. The behavioral model describes the dynamic response characteristics of the system, including control system behavior, equipment motion behavior, and material flow behavior; the control system behavior adopts a state-space model. ； Where x is the state vector, u is the input vector, y is the output vector, w and v are the process noise and measurement noise, and A, B, C, and D are the system matrices; The time derivative of the state vector; The dynamic behavior of the equipment is governed by multibody dynamics equations: ； Where q is the generalized coordinate, M is the mass matrix, C is the damping matrix, K is the stiffness matrix, and Q is the generalized force. c For binding force; It is a generalized acceleration vector; The vector is the generalized velocity vector; t represents time t. The knowledge model employs a hybrid modeling approach, combining mechanistic and data-driven models: ； Among them, f physics For the physical mechanism model, f ML This is a machine learning model, where α is the fusion weight, dynamically adjusted using an adaptive algorithm. This represents the predicted output value of the knowledge model. This is the physical model parameter vector; For machine learning model parameter vectors; ； Among them, e physics and e ML The historical errors of the physical model and the machine learning model are respectively, σ is the activation function, and w α Here, x(t) is the weight parameter vector; x(t) is the input feature at the current time step; e physics (t) represents the historical prediction error of the physical model at time t; e ML (t) represents the historical prediction error of the machine learning model at time t; b α This is the bias parameter, used to adjust the baseline value of the weights.

9. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 8, characterized in that, In the predictive simulation mode, forward simulation is performed based on the current state and boundary conditions to predict the future evolution trajectory of the system, as follows: Short-term prediction is based on time series analysis and recurrent neural networks, and a long short-term memory network is used to capture temporal dependencies. ; ; ; ； ； ； in, The hidden state at time step t; Forgotten Gate; For input gates; These are candidate values, i.e., candidate memory cells at the current moment; The cell state represents the current state of the memory cell. For output gate; The current hidden state; , , , These are the weight matrices for the forget gate, input gate, candidate values, and output gate, respectively. , , , σ represents the bias vectors for the forget gate, input gate, candidate value, and output gate, respectively; σ is the sigmoid function; ⊙ is the Hadamard product; The input represents the time step t; Using a state-space model for medium-term forecasting: ； Where, x t Let x be the state vector at time t; t+1 Let u be the state vector at the next time step (t+1); t Let y be the control input vector at time t; t Let A be the observed output vector at time t; B be the state transition matrix; C be the control input matrix; D be the output matrix; and W be the feedforward matrix. t v is the process noise vector; t For the observed noise vector; The parameter matrix is ​​obtained through system identification: ； Where θ is the parameter vector to be estimated; y represents the optimal estimate of the parameter; N is the total length of the observed data; t The actual observed value at time t; These are model predictions based on parameter θ; λ represents the square norm of the prediction error; λ is the regularization coefficient.

10. The multi-parameter collaborative optimization control method for impregnated paper production process based on reinforcement learning according to claim 9, characterized in that, The multi-objective optimization decision framework achieves collaborative optimization at different time scales, as detailed below: Production process optimization involves multiple conflicting objectives such as quality, cost, efficiency, and energy consumption. A multi-objective optimization method is used to find the Pareto optimal solution set. minF(x)=[f1(x),f2(x),f3(x),f4(x)] T ; subject to: g(x)≤0,h(x)=0; Where f1(x) is the quality index, f2(x) is the production cost, f3(x) is the energy consumption index, f4(x) is the production efficiency; g(x) represents the maximum load of the equipment; and h(x) represents the production quota. Reinforcement learning adaptive control employs a deep deterministic policy gradient algorithm to learn the adaptive control policy, where the Actor network is updated: ； in, This represents the parameter vector of the Actor network; For the deterministic policy function; s t Let be the state at time t; 'a' be the action variable. The state distribution represents the behavioral strategy; β represents the behavioral strategy. For Critic network; s represents the state; This represents the gradient of the Q-function with respect to the action. These are the parameters of the Critic network; This represents the integration over the state under the behavioral policy distribution; J is the objective function; Critic Network Update: ； Among them, y t TD target value; For experience replay buffer; r t For the immediate reward obtained; s t+1 The next state; Target network soft update: ； in, Target Critic network parameters; Here are the target Actor network parameters; τ is the soft update coefficient. Based on the predictive capabilities of digital twin models, rolling optimization is achieved using nonlinear model predictive control: ； ； Where L(x,u) is the stage cost function, V f (x) is the terminal cost function, N is the prediction time domain length; k is the time step index within the prediction time domain; f(x) k ,u k ) is the system state transition function; To control the set of input constraints; For the set of state constraints; For the terminal constraint set; x k ,u k Let these represent the system state vector and control input vector at step k, respectively; Cost function Designed as a weighted multi-objective form: ； Where w1, w2, w3 are weight coefficients, Q, R, S are weight matrices, Δu is the control increment, and u ref To control the reference value vector, x ref This is a vector of state reference values.

Images