Optimal set point tracking control method for injection molding system based on output feedback

Abstract

The application provides an injection molding system optimal set point tracking control method based on output feedback, relates to the automatic control technical field of injection molding systems, and comprises the following steps: establishing a discrete time state space equation of the injection molding system, which at least comprises a state vector and an output vector; subtracting the output vector from the corresponding set point in a reference trajectory to obtain a tracking error, and constructing an augmented system according to the state vector and the tracking error; constructing an infinite domain cost function based on the augmented system; obtaining a non-minimal state vector based on the augmented system by using a state parameterization method, and defining an output feedback Q function based on the vector; collecting input and output data of the injection molding system to design a zero-error controller, and taking the controller as an initial stable control strategy to execute an output feedback off-policy Q learning algorithm; and through iterative solving, the output feedback Q function is converged to obtain an optimal output feedback control strategy that minimizes the infinite domain cost function, thereby improving control precision.

Description

Technical Field

[0001] This application relates to the field of automatic control technology for injection molding systems, and in particular to an optimal setpoint tracking control method for injection molding systems based on output feedback. Background Technology

[0002] Setpoint tracking control is a typical control task in practical control systems. Taking injection molding as an example, this process mainly includes three stages: filling, holding (or compression / holding), and cooling. In the holding stage, the nozzle pressure is a critical process variable, and accurate setpoint tracking of this key process variable is crucial to ensuring the quality of the final product. Traditional setpoint tracking control methods obtain the optimal control strategy (or controller) by minimizing a cost function consisting of tracking error and control input. However, when the reference trajectory cannot asymptotically converge to zero, the integral of the cost function in the infinite time domain tends to infinity, causing the problem definition to fail. To solve this problem, a common practice in existing technologies is to introduce a discount factor into the cost function to ensure its boundedness. However, the stability of this method heavily depends on the selection of the discount factor. If its value is inappropriate, it may affect the stability of the closed-loop system. In particular, most existing methods do not deeply analyze the impact of the discount factor on stability, but simply conservatively choose a large value, which limits their reliability in practical industrial applications.

[0003] On the other hand, the injection molding process is characterized by strong nonlinearity, time-varying nature, and complex internal mechanisms. Coupled with factors such as material inhomogeneity and valve nonlinearity, it is difficult to establish accurate mathematical models. This makes many model-based optimal control methods (such as linear quadratic regulators, LQR) difficult to apply in practice. In recent years, model-independent data-driven control methods have received widespread attention. These methods can be mainly divided into two categories: indirect and direct. Indirect methods (such as system identification based on least squares, neural networks, etc.) first use measurement data to determine the system model, and then design the controller based on the model. However, errors and uncertainties in the model determination stage will be transmitted to the controller, affecting the final performance. Direct methods bypass the modeling step and directly use data to design the controller. Representative methods include reinforcement learning and data-driven behavioral systems theory.

[0004] Q-learning, as a mainstream model-free reinforcement learning algorithm, is widely used in optimal control problems. For the infinite-domain tracking problem, some researchers have proposed a Q-learning algorithm that can solve the augmented algebraic Riccati equations online. For nonlinear systems, some researchers have used neural networks to approximate the Q-function. However, all these methods use cost functions with discount factors, and therefore also face stability risks caused by inappropriate selection of discount factors.

[0005] To circumvent the problem of inappropriate discount factor selection, some scholars have introduced relative and incremental control variables into the cost function and designed corresponding Q-learning algorithms. However, this method relies on the strong assumption that all system states are measurable. In actual injection molding systems, due to harsh conditions such as high temperature, high pressure, and enclosed environments, accurately measuring all states is extremely costly or difficult to achieve, thus limiting the applicability of the aforementioned full-state feedback-based methods. Of course, existing data-driven control methods based on behavioral systems theory can also circumvent the problem of inappropriate discount factor selection, but these methods also require knowledge of all system states.

[0006] Therefore, for multivariable processes such as injection molding systems where the state of the model is not fully known, how to design an optimal setpoint tracking strategy that does not rely on discount factors and does not require determining all states of the system model remains an important technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] This application provides an optimal setpoint tracking control method for injection molding systems based on output feedback, aiming to achieve optimal setpoint tracking without relying on discount factors and precise system models. The technical solution is as follows:

[0008] This application provides an optimal setpoint tracking control method for an injection molding system based on output feedback, the method comprising:

[0009] Establish the discrete-time state-space equation of the injection molding system, wherein the discrete-time state-space equation includes a state vector, an input vector, and an output vector;

[0010] The tracking error is obtained by subtracting the output vector from the corresponding set point in the reference trajectory, and an augmented system is constructed based on the state vector and the tracking error.

[0011] Based on the augmented system, an infinite-domain cost function without discount factors is constructed;

[0012] Using the state parameterization method, the augmented state vector of the augmented system is equivalently represented as a non-minimum state vector composed of historical input and output data;

[0013] Based on the aforementioned non-minimum state vector, define the output feedback Q function;

[0014] Input and output data of the injection molding system are collected to design a deadbeat controller. The deadbeat controller is used as the initial stable control strategy to execute the output feedback policy Q learning algorithm. The output feedback Q function is converged through iterative solution to obtain the optimal output feedback control strategy that minimizes the infinite domain cost function.

[0015] In one possible implementation, constructing an augmented system based on the state vector and the tracking error includes:

[0016] Calculate the difference between the state vectors at adjacent time points;

[0017] An augmented state vector is defined based on the state vector difference and the tracking error;

[0018] An augmented system is obtained based on the augmented state vector.

[0019] In one possible implementation, the augmented state vector is:

[0020] ;

[0021] in, This represents the difference in the state vectors, and , express The state vector at time t, express The state vector at any given time; Indicates the tracking error, and , , They represent in The output vector and setpoint at time t. , These are the dimensions of the state vector and the output vector, respectively.

[0022] In one possible implementation, the augmentation system is:

[0023] ;

[0024] in, , , This represents the augmented state vector. This represents the increment of the input vector. 、 、 These represent the system matrix, input matrix, and output matrix of the injection molding system, respectively. The dimension is The identity matrix, , , These represent the dimensions of the vector or matrix, respectively.

[0025] The input to the augmentation system is the increment of the input vector. The state of the augmented system is determined by the difference between the state vectors. and tracking error The constructed augmented state vector When the augmented state vector When adjusted to zero, the injection molding system can achieve steady state and optimal setpoint tracking.

[0026] In one possible implementation, the state parameterization method is used to equivalently represent the augmented state vector of the augmented system as a non-minimum state vector composed of historical input and output data, including:

[0027] Based on the state parameterization method, the state parameterization of the augmented state vector is obtained, and the state parameterization of the augmented state vector includes the non-minimum state vector.

[0028] In one possible implementation, the state parameterization of the augmented state vector is as follows:

[0029] ;

[0030] in, This represents the non-minimum state vector. Represents the non-minimum state Increment along the time direction, This indicates the observable parameters of the injection molding system. It is a full-rank row transformation matrix that depends on the model parameters. .

[0031] In one possible implementation, based on the non-minimum state vector, an output feedback Q function is defined, including:

[0032] The value function is determined based on the optimal state feedback control strategy of the augmented system.

[0033] Construct a state feedback Q function based on the value function;

[0034] The output feedback Q function is constructed based on the state parameterization of the augmented state vector and the state feedback Q function.

[0035] In one possible implementation, the output feedback Q-function is constructed based on the state parameterization of the augmented state vector and the state feedback Q-function, including:

[0036] Substituting the state parameterization of the augmented state vector into the state feedback Q function, we obtain:

[0037] ;

[0038] in, It is the auxiliary vector of the state feedback Q function. It is an auxiliary vector used to construct the output feedback Q function. , The dimension is The identity matrix;

[0039] Based on the auxiliary vector, the output feedback Q function is obtained as follows:

[0040] ;

[0041] in, , , These represent the parameter matrices of the output feedback Q-function and the state feedback Q-function, respectively. , , These represent the block matrices of the output feedback Q-function parameter matrix, , , , These represent the block matrices of the state feedback Q-function parameter matrix, where, It is a solution to the algebraic Riccati equation of the augmented system. , , This represents a pre-defined weight matrix.

[0042] In one possible implementation, the method further includes:

[0043] After obtaining the output feedback Q function, the optimal output feedback control strategy is obtained by minimizing the output feedback Q function.

[0044] In one possible implementation, the deadbeat controller is used as the initial stable control strategy to execute an output feedback policy Q-learning algorithm. The output feedback Q-function is converged through iterative solving to obtain the optimal output feedback control strategy that minimizes the infinite domain cost function, including:

[0045] The input and output data of the injection molding system are processed to obtain a non-singular matrix. Z sum matrix W ;

[0046] Multiply by the left side of the state feedback Q function Multiply by the right This yields a parameter matrix that includes the output feedback Q function. The equation to be solved: ;in, , , This represents the gain of the output feedback Q function;

[0047] If the equation to be solved requires iteration Then the equation to be solved is: ; and in the first iteration of solving the equation to be solved, let And select an initial stability control strategy that asymptotically stabilizes the injection molding system from its initial state;

[0048] Iterative solution of the equation to be solved In order to continuously implement old strategies Update to a new control strategy , ;

[0049] when At the time of its establishment, The value is a preset constant. The program execution ends and the optimal output feedback control strategy of the injection molding system is output.

[0050] The technical solutions provided in this application can achieve the following technical effects:

[0051] (1) On the one hand, the augmented state vector of this application includes the state vector difference. and tracking error Two components, each corresponding to an independent control objective: when component When the state vector no longer changes at the preceding and following moments, it indicates that the injection molding system has entered a steady state; when the components This indicates that the output vector is consistent with the corresponding setpoint in the reference trajectory, meaning that the pressure at the nozzle of the injection molding system has reached the optimal setpoint tracking. Furthermore, an augmented system is generated based on the augmented state vector, which defines its input as the increment of the input vector. And its state is defined as an augmented state vector. This means that the control objective of the injection molding system is no longer tracking the reference trajectory, but rather the augmented state vector. By adjusting to zero, the complex setpoint tracking problem is successfully transformed into a typical regulation problem. Simultaneously, due to the augmented state vector... When adjusted to zero, the injection molding system is in a steady state, so the integrand in the cost function eventually decays to zero, thus ensuring the convergence of the infinite integral and completely eliminating the dependence on the discount factor.

[0052] (2) On the other hand, based on the state parameterization method, this application first represents the augmented state vector of the augmented system as a non-minimum state vector composed of historical input data and output data, and then constructs the output feedback Q function and designs the optimal output feedback control strategy based on this non-minimum state vector, so that this application does not need to rely on all the state measurement information of the injection molding system, that is, it does not need to know all the states of the injection molding system, and can also achieve optimal setpoint tracking.

[0053] (3) Furthermore, this application utilizes the input and output data generated during the operation of the injection molding system to design a deadbeat controller based on the input and output data. This controller is used as the initial stable control strategy for the output feedback off-policy Q-learning algorithm. The optimal output feedback control strategy is then obtained by executing the output feedback off-policy Q-learning algorithm. Thus, it can be seen that the entire solution process does not require prior modeling, yet it can still guarantee that the solution is the optimal output feedback control strategy. In terms of closed-loop stability, tracking accuracy, and anti-interference capability, it achieves the performance of the full-state feedback control method based on the accurate model. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. In the drawings:

[0055] Figure 1 This is a flowchart of the optimal setpoint tracking control method for an injection molding system based on output feedback, according to an embodiment of this application.

[0056] Figure 2 This is a schematic diagram of the optimal setpoint tracking control principle in the embodiments of the method of this application;

[0057] Figure 3 This is a flowchart of the execution output feedback policy Q-learning algorithm in the method embodiment of this application;

[0058] Figure 4 This is the output curve trajectory of the injection molding system in the method embodiment of this application under undisturbed conditions;

[0059] Figure 5 This is the output curve trajectory of the injection molding system in the method embodiment of this application under unknown constant disturbance conditions;

[0060] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0061] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0063] This application provides an optimal setpoint tracking control method for an injection molding system based on output feedback, such as... Figure 1 As shown, the method mainly includes the following steps S101 to S106.

[0064] Step S101: Establish the discrete-time state-space equation of the injection molding system. The discrete-time state-space equation includes the state vector, input vector, and output vector.

[0065] Injection molding systems typically refer to pressure-holding process control systems that use the opening of hydraulic control valves as the control input and nozzle pressure as the controlled output. In this embodiment, the system specifically refers to the dynamic process from control input to process output. Controlling such systems faces two major challenges: the inability to directly measure internal states and the difficulty in establishing accurate mathematical models due to complex mechanisms. To overcome these challenges, this embodiment abandons the traditional approach of precise modeling and instead utilizes discrete-time state-space equations as the theoretical framework for describing the dynamic characteristics of such systems, thereby providing data support for subsequent data-driven control methods that are completely independent of model parameters.

[0066] Specifically, the formula for calculating the discrete-time state-space equation is as follows:

[0067] (1)

[0068] Among them, subscript Indicates time, It is the set of natural numbers. , , These are the state vector, input vector, and output vector of the injection molding system. This represents the initial state of the injection molding system. express The real space of dimension 1 , , These are the dimensions of the state vector, input vector, and output vector, respectively. 、 、 It is a coefficient matrix with the corresponding dimension, specifically, 、 、 These represent the system matrix, input matrix, and output matrix of the injection molding system, respectively.

[0069] The aforementioned state vector refers to a set of variables that describe the internal dynamics of the injection molding system and are typically not directly measurable, such as melt pressure and temperature. The input vector refers to the opening degree of the hydraulic control valve. The output vector refers to a physical quantity that can be directly measured by sensors and reflects the key performance of the injection molding system; in this embodiment, it specifically refers to the pressure at the nozzle. The state vector, input vector, and output vector together constitute the dynamic causal chain of the injection molding system: by changing the input vector (the opening degree of the hydraulic control valve), the injection molding system is driven, causing a change in its state vector (such as melt pressure and temperature). This internal change is ultimately perceived as a change in the output vector (the pressure at the nozzle).

[0070] Step S102: Subtract the output vector from the corresponding set point in the reference trajectory to obtain the tracking error, and construct an augmented system based on the state vector and the tracking error.

[0071] First, subtract the output vector from the corresponding setpoint in the reference trajectory to obtain the tracking error:

[0072] (2)

[0073] in, , They represent in The output vector and setpoints at each moment, with multiple setpoints arranged in chronological order, form a reference trajectory. For example... Figure 2 As shown, to achieve optimal setpoint tracking in this embodiment, it is necessary to ensure that the output vector is consistent with the corresponding setpoint in the reference trajectory. That is, the pressure at the nozzle of the injection molding system is the same as the pre-set pressure value. After a period of time, the curve formed by the pressure at the nozzle is the same as the reference trajectory and covers the reference trajectory, thereby achieving optimal setpoint tracking of the pressure at the nozzle.

[0074] Then, calculate the state vector difference between adjacent time steps:

[0075] (3)

[0076] in, express The state vector at time t, express The state vector at time step, therefore It can also be called a state increment.

[0077] Based on the obtained state vector difference and tracking error Define the augmented state vector:

[0078] (4)

[0079] It should be noted that all superscript "T" appearing in this application for any parameter indicates the transpose symbol.

[0080] The augmented state vector described above is derived from the state vector difference. and tracking error A new vector is formed by these components, defined such that each of the two components corresponds to an independent control objective: when the components... When the state vector no longer changes at the preceding and following moments, it indicates that the injection molding system has entered a steady state; when the components When the output vector is adjusted to zero, it indicates that the output vector is consistent with the corresponding setpoint in the reference trajectory, meaning that the pressure at the nozzle of the injection molding system has reached the optimal setpoint tracking. Therefore, when the augmented state vector is adjusted to zero, it indicates that the injection molding system has reached a steady state and simultaneously achieved optimal setpoint tracking.

[0081] Based on the augmented state vector, the augmented system is obtained:

[0082] (5)

[0083] in, , , The dimension is The identity matrix.

[0084] As can be seen from the above formula (5), the augmented system defines its input as the increment of the input vector. And its state is defined by the difference between the state vectors. and tracking error The constructed augmented state vector This means that the control objective of the injection molding system is no longer tracking the reference trajectory, but rather the augmented state vector. Adjust to zero, because when the augmented state vector When adjusted to zero, not only can the injection molding system reach a steady state, but also optimal setpoint tracking can be achieved. Therefore, this embodiment can successfully transform the complex setpoint tracking problem into a typical adjustment problem.

[0085] Step S103: Based on the augmented system, construct an infinite-domain cost function without discount factors.

[0086] Based on the augmented system described above, the problem of infinite domain optimization can also be fundamentally solved because: the adjustment target (augmented state vector) in this embodiment Adjust to zero, that is This is achieved naturally in steady state, causing the integrand in the cost function to eventually decay to zero, thus ensuring the convergence of the infinite integral and completely eliminating the dependence on the discount factor.

[0087] In this embodiment, the formula for calculating the infinite field cost function is:

[0088] (6)

[0089] in, , , The weight matrix is ​​set in advance.

[0090] The infinite-domain cost function comprehensively considers all tracking performance and control costs from the current moment to the future through an integral form. Its minimum value corresponds to the theoretically globally optimal control performance. This function directly determines the optimization direction of the subsequent output feedback Q function, guiding the Q function to find a control strategy that minimizes the long-term cumulative cost, thereby ensuring that the closed-loop system achieves accurate tracking of the setpoint while operating stably.

[0091] Step S104: Using the state parameterization method, the augmented state vector of the augmented system is equivalently represented as a non-minimum state vector composed of historical input data and output data.

[0092] As can be seen from the state parameterization method, the system state can be equivalently described by historical input and output data, as well as a preset parameter matrix, that is:

[0093] (7)

[0094] in, It is a defined non-minimum state. Indicates from the previous A vector composed of the inputs at each time step. It can be obtained by calculating observable indicators. It is a transformation matrix calculated based on the system's input and output data. It is a full-rank row transformation matrix that depends on the model parameters, and its specific parameters satisfy:

[0095] (8)

[0096] in, It is an invertible transformation matrix. Include of Linearly independent rows Include The remaining lines, and The composition method is the same as and , It can be obtained by asking inverse matrix To obtain, among which .

[0097] Based on the state parameterization method described above, the state parameterization of the augmented state vector in this embodiment can be expressed as:

[0098] (9)

[0099] in, It is the non-minimum state vector of the augmented system. Represents the non-minimum state Increment along the time direction, This indicates observable metrics of the injection molding system.

[0100] Since the state vector of an injection molding system cannot be directly measured, and the non-minimum state vector is a vector formed by stacking the input and output data of the injection molding system over a period of time, we can use the non-minimum state vector and construct its equivalence relationship with the theoretical augmented state vector through the state parameterization method. This will lay the foundation for designing the optimal output feedback control strategy based entirely on the input and output data.

[0101] Step S105: Define the output feedback Q function based on the non-minimum state vector.

[0102] First, construct the state feedback Q function.

[0103] According to standard linear quadratic regulation theory, the optimal state feedback control strategy for an augmented system is:

[0104] (10)

[0105] For the feedback gain in any steady state At a given time The corresponding value function It is a quadratic form, specifically:

[0106] (11)

[0107] in, It is feedback gain The value of the function parameter.

[0108] According to the Bellman optimality principle, the value function (11) above can be written in the following recursive form:

[0109] (12)

[0110] Combining formulas (11) and (12), construct the state feedback Q function:

[0111] (13)

[0112] Where, vector It is the auxiliary vector of the state feedback Q-function, a matrix Let be the parameter matrix of the state feedback Q function and , , , , These represent the block matrices of the state feedback Q-function parameter matrix. It is a solution to the augmented system algebraic Riccati equation.

[0113] Specifically, the optimal state feedback control strategy (10) of the augmented system can be obtained by minimizing the state feedback Q function (13), that is:

[0114] (14)

[0115] Due to the equivalence of formulas (12) and (13), the state feedback Q function can be further written in the following recursive form:

[0116] (15)

[0117] Furthermore, formula (15) can be written in the following simplified form:

[0118] (16)

[0119] Substitute control strategy Formula (16) can also be equivalently described as the following matrix equation:

[0120] (17)

[0121] in, , .

[0122] From formula (14), it can be seen that the optimal state feedback gain The matrix in formula (17) can be solved. get.

[0123] The aforementioned state feedback Q-function measures the long-term total cost of an injection molding system after taking a certain action in a specific state. For example, based on the current augmented state vector of the injection molding system, when preparing to execute a certain control action (including increasing, decreasing, or keeping the input vector unchanged), the function will give a score. This score represents the sum of all future costs accumulated from the current augmented state vector until the optimal control action is executed. In other words, the state feedback Q-function not only considers the immediate cost of the current control action, but more importantly, it inherently predicts and includes all the future impacts of the control action. The optimal state feedback control strategy is obtained by minimizing the state feedback Q-function; that is, the optimal state feedback control strategy represents what the optimal control action should be taken in any given control action selection process.

[0124] Secondly, the output feedback Q function is constructed. Specifically, the auxiliary vector of the output feedback Q function is constructed first by parameterizing the augmented state vector and the state feedback Q function. That is, by substituting the state parameterization of the augmented state vector into the calculation formula (13) of the state feedback Q function, we get:

[0125] (18)

[0126] in, It is an auxiliary vector used to construct the output feedback Q function. , The dimension is The identity matrix.

[0127] Based on the constructed auxiliary vector, the output feedback Q function is obtained:

[0128] (19)

[0129] in, , This represents the parameter matrix of the output feedback Q-function. , , These represent the block matrices of the output feedback Q-function parameter matrix.

[0130] The aforementioned output feedback Q function is a Q function whose input consists entirely of measurable data. It can evaluate and find the optimal control action (also known as the optimal control strategy) based solely on the input and output data without relying on the measurement of any state vector. Therefore, in this embodiment, after obtaining the output feedback Q function, the optimal output feedback control strategy is obtained by minimizing the output feedback Q function.

[0131] Specifically, the process of obtaining the optimal output feedback control strategy is as follows:

[0132] First, the state parameterization of the augmented state vector is substituted into the optimal state feedback control strategy to obtain the initial optimal output feedback control strategy:

[0133] (20)

[0134] in, , .

[0135] Based on the initial optimal output feedback control strategy described above, the final optimal output feedback control strategy is obtained by minimizing the output feedback Q function:

[0136] (twenty one)

[0137] Among them, by changing both sides of the initial optimal output feedback control strategy from 0 to... Summing the results, we obtain the final optimal output feedback control strategy of the injection molding system in proportional-integral form:

[0138] (twenty two)

[0139] Step S106: Collect the input and output data of the injection molding system to design a deadbeat controller, and use the deadbeat controller as the initial stable control strategy to execute the output feedback policy Q learning algorithm. Through iterative solution, the output feedback Q function is converged to obtain the optimal output feedback control strategy that minimizes the infinite domain cost function.

[0140] First, a deadbeat-free controller is designed based on offline data as the initial stable control strategy. The application order for the injection molding system is... The continuous excitation input signal is used to collect the input vector, output vector, and tracking error generated during the operation of the injection molding system, resulting in a set of lengths of... N The data sequence is defined as a matrix as follows:

[0141] (twenty three)

[0142] make Construct the following non-singular square matrix sum matrix :

[0143] , (twenty four)

[0144] Multiply the formula (17) on the left right multiplication This yields a parameter matrix that includes the output feedback Q function. The equation to be solved:

[0145] (25)

[0146] in, , , This represents the gain of the output feedback Q function.

[0147] Based on the data matrix in the above formula (23) , and The deadbeat controller is designed through the following steps a1 to a5:

[0148] Step a1: Through the data matrix , and Define the matrix of the virtual system and ;

[0149] Step a2: Let for Moore-Penrose pseudo-reverse, for The basis of the null space is defined as follows: the matrix of the virtual system is... , ;

[0150] Step a3: Definition For a set of linearly independent matrices The full column rank matrix formed by the columns of is such that ;

[0151] Step a4: Use the transformation matrix Virtual system Transform into the canonical form of a multiple-input multiple-output controller. ;

[0152] Step a5: For the system Design a time-lapse controller , making It is zero-power.

[0153] Based on the obtained deadbeat controller By calculating the matrix and in the matrix Add zero rows to construct a new matrix. The position where the zero line is added is the same as The middle is formed The columns that are removed are in the same position, making Then, through calculation of the deadbeat controller... Gain , recorded as In order to gain this The deadbeat controller This serves as the initial stable control strategy for the subsequent output feedback policy Q-learning algorithm.

[0154] The output feedback Q-learning algorithm is a reinforcement learning method that iteratively solves for the optimal output feedback control strategy using the system's operational data without requiring prior modeling of the injection molding system. The core principle of this method is that through continuous iteration of policy evaluation and improvement, the parameters of the output feedback Q-function converge, thereby obtaining an optimal output feedback control strategy that minimizes the infinite-domain cost function.

[0155] like Figure 3 As shown, the process of executing the output feedback Q-learning algorithm in this embodiment is as follows: steps b1 to b6:

[0156] Step b1: Collect the input and output data of the injection molding system, and process the collected data according to formulas (23) and (24) to obtain a non-singular matrix. sum matrix .

[0157] Step b2: Initialize the injection molding system, i.e., solve the equation to be solved in the iterative solution formula (25). (Assuming iteration) Next, then When ), first order At the same time, an initial stability control strategy is selected to make the injection molding system asymptotically stable from the initial state.

[0158] Step b3: Iterative solution Because of the parameter matrix in the equation to be solved Derived from the output feedback Q function, therefore, for the first... The control strategy used in the next iteration yields a corresponding parameter matrix. , the parameter matrix Substituting the values ​​into the output feedback Q function, we obtain a score for the output feedback Q function, which is used to evaluate the first... In the next iteration, is there room for improvement in the control strategy used, so as to continuously optimize the selected control strategy?

[0159] Step b4: In the iterative solution of the equation to be solved in step b3, the Q-function outputs a score in each iteration. Based on this score, the selected control strategy is further optimized, i.e., the old strategy is used. It can drive the output feedback Q function to output a score, and this score can then be used to generate a new control strategy. .

[0160] Step b5: During the continuous optimization of the selected control strategy (iterative output feedback Q function), if Established, And it is a very small constant, so the program execution ends. In other words, the control strategy is continuously selected by the score output by the output feedback Q function until the selected control strategy remains basically unchanged. This indicates that the selected control strategy is the optimal control strategy (also known as the optimal output feedback control strategy).

[0161] Step b6: Output the optimal output feedback control strategy for the injection molding system:

[0162] (26)

[0163] In this embodiment, the infinite domain cost function is a standard for measuring the long-term performance of the injection molding system. However, directly optimizing this function is difficult. The output feedback Q function is actually another manifestation of this cost function, similar to a "real-time scoring system." Therefore, this embodiment drives the output feedback Q function to converge through an output feedback policy Q-learning algorithm. Essentially, this algorithm makes the Q function gradually approach and eventually satisfy its corresponding Bellman optimality equation. Mathematically, the solution to this equation and the global minimum solution of the infinite domain cost function are mutually necessary and sufficient conditions. Therefore, when the algorithm converges iteratively, the optimal output feedback control strategy derived from the output feedback Q function is the global optimal solution under the definition of the infinite domain cost function, thereby ensuring that the injection molding system meets the two objectives of stability and optimal setpoint tracking.

[0164] Therefore, this embodiment adopts a fully data-driven approach. First, it uses offline data to design an initial deadbeat controller that ensures the stability of the injection molding system. Then, based on this, it iteratively solves the optimal output feedback control strategy that is consistent with the global optimal solution of the infinite domain cost function by executing the output feedback strategy Q-learning algorithm. The entire process does not require any prior knowledge about the system model.

[0165] To illustrate the effectiveness of the optimal setpoint tracking control method for injection molding systems based on output feedback proposed in this application, a specific injection molding system is used as an example below:

[0166] First, based on open-loop testing and analysis, the pressure at the nozzle as a function of the hydraulic control valve opening is determined to be the following discrete-time state-space equation:

[0167] (27)

[0168] Wherein, the input vector is the opening degree of the hydraulic control valve, the output vector is the pressure at the nozzle, and the initial state vector is set to... , , , The weight matrix is , The set point is... , , Let be a set of integers. The random input signal that satisfies the continuous excitation condition. Applying this to the above formula (27) and collecting data, the collected offline input and output data are preprocessed to obtain a sequence with a length of [missing information]. The incremental input data and incremental output data are respectively and and tracking error is .make Construct a non-singular matrix sum matrix The Q-learning algorithm for the output feedback policy is executed. During execution, after 10 iterations, the gain of the deadbeat controller is obtained as follows: The numerical result is exactly the same as the optimal solution obtained using the MATLAB idare function based on the exact model, verifying the effectiveness of the algorithm.

[0169] For the injection molding system in the above example, the output curve when the system is free from disturbance is as follows: Figure 4 As shown, when a disturbance is added to the system ( and In the case of ), the output curve of the injection molding system is as follows: Figure 5 As shown. By Figure 4 and Figure 5 The results show that the optimal setpoint tracking control method for injection molding system based on output feedback in this embodiment can achieve optimal tracking of the setpoint of the reference trajectory. In addition, this method also has a good suppression effect on unknown constant disturbances.

[0170] Furthermore, for the injection molding system in the above example, when there is no disturbance in the system, the tracking performance (integral absolute error and mean square error) of the control method of this application is compared with that of the model-based optimal strategy and the strategy with discount factor. The comparison results are shown in Table 1.

[0171] Table 1

[0172]

[0173] As shown in Table 1, the control method of this application achieves almost the same control accuracy as the model-based optimal strategy, while the control performance of the strategy with discount factor is the worst. Furthermore, the control method of this application is entirely data-driven, while model-based solutions require precisely known model information.

[0174] Furthermore, this embodiment also compares and analyzes the performance of the control method of this application with other data-driven methods in solving the optimal strategy, and calculates the statistical results (average error and average running time) based on 100 independent experiments and recorded experimental data, as shown in Table 2.

[0175] Table 2

[0176]

[0177] As shown in Table 2, the control method of this application is superior to reinforcement learning and semidefinite programming methods in terms of both computational accuracy and efficiency.

[0178] The results above demonstrate that the optimal setpoint tracking control method for injection molding systems based on output feedback proposed in this application can achieve optimal steady-state tracking of the reference trajectory even when the system model information and all state measurement information are unknown. Furthermore, this control strategy also has a good suppression effect on unknown constant disturbances.

[0179] It should be noted that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In practical applications, all the above possible implementation methods can be arbitrarily combined in a combined manner to form possible embodiments of this application, which will not be described in detail here.

[0180] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the optimal setpoint tracking control method for injection molding systems based on output feedback of any of the above embodiments.

[0181] In an exemplary embodiment, an electronic device is provided, such as Figure 6 As shown, Figure 6 The illustrated electronic device 600 includes a processor 601 and a memory 603. The processor 601 and the memory 603 are connected, for example, via a bus 602. Optionally, the electronic device 600 may also include a transceiver 604. It should be noted that in practical applications, the transceiver 604 is not limited to one type, and the structure of this electronic device 600 does not constitute a limitation on the embodiments of this application.

[0182] Processor 601 may be a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 601 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0183] Bus 602 may include a pathway for transmitting information between the aforementioned components. Bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 602 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0184] The memory 603 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0185] The memory 603 stores computer program code that executes the scheme of this application, and its execution is controlled by the processor 601. The processor 601 executes the computer program code stored in the memory 603 to implement the content shown in the foregoing method embodiments.

[0186] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0187] Based on the same inventive concept, this application also provides a storage medium storing a computer program, wherein the computer program is configured to execute the optimal setpoint tracking control method for injection molding systems based on output feedback of any of the above embodiments when running.

[0188] Those skilled in the art will clearly understand that the specific working process of the systems, devices, and modules described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0189] Those skilled in the art will understand that the technical solution of this application, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several program instructions to cause an electronic device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of this application when running the program instructions. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0190] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as electronic devices like personal computers, servers, or network devices) associated with program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of this application.

[0191] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of this application, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to leave the protection scope of this application.

Claims

1. A method for tracking and controlling the optimal setpoint of an injection molding system based on output feedback, characterized in that, include: A discrete-time state-space equation for the injection molding system is established, comprising a state vector, an input vector, and an output vector; the calculation formula for the discrete-time state-space equation is as follows: Among them, subscript Indicates time, It is the set of natural numbers. , , These are the state vector, input vector, and output vector of the injection molding system. This represents the initial state of the injection molding system. express The real space of dimension 1 、 、 It is a coefficient matrix with the corresponding dimension, specifically, 、 、 These represent the system matrix, input matrix, and output matrix of the injection molding system, respectively. , , These represent the dimensions of the vector or matrix, respectively. The tracking error is obtained by subtracting the output vector from the corresponding set point in the reference trajectory, and an augmented system is constructed based on the state vector and the tracking error. Based on the augmented system, an infinite-domain cost function without discount factors is constructed; Using the state parameterization method, the augmented state vector of the augmented system is equivalently represented as a non-minimum state vector composed of historical input and output data; Based on the aforementioned non-minimum state vector, define the output feedback Q function; The input and output data of the injection molding system are collected to design a deadbeat controller. The deadbeat controller is used as the initial stable control strategy to execute the output feedback policy Q learning algorithm. The output feedback Q function is converged through iterative solution to obtain the optimal output feedback control strategy that minimizes the infinite domain cost function. Constructing an augmented system based on the state vector and the tracking error includes: Calculate the difference between the state vectors at adjacent time points; An augmented state vector is defined based on the state vector difference and the tracking error; The augmented system is obtained based on the augmented state vector; The augmented state vector is: ； in, This represents the difference in the state vectors, and , express The state vector at time t, express The state vector at any given time; Indicates the tracking error, and , Indicates in The set point of time, , These are the dimensions of the state vector and the output vector, respectively. The augmentation system is: ； in, , , This represents the increment of the input vector. The dimension is The identity matrix; The input to the augmentation system is the increment of the input vector. The state of the augmented system is determined by the difference between the state vectors. and tracking error The constructed augmented state vector When the augmented state vector When adjusted to zero, the injection molding system can achieve steady state and optimal setpoint tracking; The formula for calculating the infinite field cost function is as follows: in, , , The weight matrix is ​​set in advance; Using the state parameterization method, the augmented state vector of the augmented system is equivalently represented as a non-minimum state vector composed of historical input and output data, including: Based on the state parameterization method, the state parameterization of the augmented state vector is obtained, and the state parameterization of the augmented state vector includes the non-minimum state vector; The state parameterization of the augmented state vector is as follows: ； in, This represents the non-minimum state vector. Represents the non-minimum state Increment along the time direction, This indicates the observable parameters of the injection molding system. It is a full-rank row transformation matrix that depends on the model parameters. ; Based on the aforementioned non-minimum state vector, an output feedback Q function is defined, including: The value function is determined based on the optimal state feedback control strategy of the augmented system; the value function is: in, It is feedback gain The value of the function parameter; According to the Bellman optimality principle, the recursive form of the value function is: A state feedback Q-function is constructed based on the value function; the state feedback Q-function is: Where, vector It is the auxiliary vector of the state feedback Q-function, a matrix Let be the parameter matrix of the state feedback Q function and , , , , These represent the block matrices of the state feedback Q-function parameter matrix; Construct an output feedback Q function based on the state parameterization of the augmented state vector and the state feedback Q function; Based on the state parameterization of the augmented state vector and the state feedback Q function, an output feedback Q function is constructed, including: Substituting the state parameterization of the augmented state vector into the state feedback Q function, we obtain: ； in, It is an auxiliary vector used to construct the output feedback Q function. , The dimension is The identity matrix; Based on the auxiliary vector, the output feedback Q function is obtained as follows: ； in, , This represents the parameter matrix of the output feedback Q-function. , , These represent the block matrices of the output feedback Q-function parameter matrix, It is a solution to the algebraic Riccati equation of the augmented system; The method further includes: After obtaining the output feedback Q function, the optimal output feedback control strategy is obtained by minimizing the output feedback Q function; Using the deadbeat controller as the initial stable control strategy, the output feedback Q-learning algorithm is executed. Through iterative solving, the output feedback Q-function converges to obtain the optimal output feedback control strategy that minimizes the infinite domain cost function, including: The input and output data of the injection molding system are processed to obtain a non-singular matrix. Z sum matrix W ; wherein, the non-singular square matrix Z for: The matrix W for: Based on the equivalence between the recursive value function and the state feedback Q function, the recursive state feedback Q function is obtained: The simplified form of the recursive state feedback Q-function is: control strategy Substituting the simplified form of the recursive state feedback Q-function, we obtain the matrix equation: in, , ; Multiplying the left side of the matrix equation Multiply by the right This yields a parameter matrix that includes the output feedback Q function. The equation to be solved: ;in, , , This represents the gain of the output feedback Q function; If the equation to be solved requires iteration Then the equation to be solved is: ; and in the first iteration of solving the equation to be solved, let And select an initial stability control strategy that asymptotically stabilizes the injection molding system from its initial state; Iterative solution of the equation to be solved In order to continuously implement old strategies Update to a new control strategy , ; when At the time of its establishment, The program terminates and outputs the optimal output feedback control strategy of the injection molding system, which is a preset constant. The optimal output feedback control strategy is as follows: .

Images