Pareto optimal control method for linear random system with Brownian motion and exponential cost functional

Abstract

The invention provides a Pareto optimal control method for a linear random system with Brownian motion and an exponential cost functional, and the method comprises the steps: carrying out the weighting and processing of a quadratic functional under the condition that the quadratic functional in the cost functional is a convex function and an allowable control set is a convex set; and the problem is converted into a minimization problem of weighted and quadratic functional. A corresponding Riccati equation is obtained through derivation, and an optimal control strategy is obtained through a matching method.

Description

Technical Field

[0001] The present invention belongs to the technical field of linear system control, and relates to a method for solving multi-objective and multi-agent optimal control of a stochastic linear system, in particular to a Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional. Background Art

[0002] Modern industrial systems are increasingly showing the characteristics of multiple participants and multiple objectives, and there is a game relationship of competition or cooperation among their optimization objectives. Designing a control strategy that can not only ensure the interests of each participant but also maximize the overall interests has become a prominent problem in current research. Among various games, the Pareto game can achieve the optimal overall benefit and the optimal allocation of limited resources, and is widely used in finance, applied science and other social sciences, as well as the resource and service industries.

[0003] In current research, the objective function involved is mostly quadratic. However, many actual systems are affected by various risks during operation. Researchers describe it as a risk-sensitive optimal control problem through an exponential objective function. The research on risk-sensitive control problems mostly focuses on single-objective and single-agent systems, while there are also various risks in multi-objective and multi-agent systems. Summary of the Invention

[0004] The purpose of the present invention is to address the above problems in the prior art, and propose a Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional, applying Pareto optimal control to the optimal control of the exponential objective function to solve the risk-sensitive optimal control problem of multi-agent and multi-objective systems.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional, comprising the following steps:

[0006] S1. Simplify and parameterize the linear stochastic system with Brownian motion:

[0007] 1). Select the following linear stochastic system with Brownian motion:

[0008]

[0009] where x 0 is the determined initial state, (x(t), t ∈ [0, T]) is the system state vector, is the control input vector of player j {j＝1,2，…，N} , (w(t), t ∈ [0, T]) is defined on (Ω, ) A standard Wiener process with appropriate dimensions, where and T is a fixed value. The coefficients {A(t), B j (t), D(t)} are known time-varying matrices with appropriate dimensions.

[0010] 2) The exponential cost functional for mutual coupling among players is:

[0011]

[0012] Let the quadratic functional appearing in the above formula be

[0013]

[0014] where Q i (t)>0, when i≠j, R ij (t)≥0, R ii (t)>0, etc. are all known time-varying matrices with appropriate dimensions, M≥0 is a known time-invariant matrix with appropriate dimensions, and E(·) represents the mathematical expectation. Let α i >0, i = 1,..., N satisfy So exists. The weighted sum of the quadratic functional is processed to obtain

[0015]

[0016] The optimal solution of the weighted sum quadratic functional is the Pareto solution.

[0017] 3) We set the joint control rate u(t) = col(u 1 (t), u 2 (t),..., u N (t)) ∈ R m and the admissible control convex set U ad . The simplified forms of the linear stochastic system and the weighted sum quadratic functional are

[0018] dx(t) = (A(t)x(t) + B(t)u(t))dt + D(t)dw(t)

[0019] and

[0020]

[0021] where B = [B 1 (t) B 2 (t)... B N (t)], R α = diag{R 1α , R 2α ,..., R Nα}.

[0022] S2. Convexity Verification of Weighted Sum and Quadratic Functionals

[0023] 1). The following conclusion is known to show that if U ad and the quadratic objective functional are both convex functions, then the Pareto - efficient strategy can be obtained by solving the weighted - sum optimal problem.

[0024] Let α i > 0, i = 1, …, N satisfy Assume that for i = 1, …, N, U ad and the quadratic objective functional are both convex functions. If u * ∈U ad is Pareto - efficient, then for all Pareto - efficient u * there exists α such that

[0025]

[0026] It has been assumed previously that U ad is convex. Considering that the objective functions involved are exponential - type cost functionals and include quadratic functionals, the convexity of the quadratic functional needs to be verified.

[0027] 2). Prove the convexity of L i when i = 1. Let x u be the state trajectory through the input variable u, and assume u, v ∈ U ad , so we infer from the convexity of U ad that for all λ ∈ (0, 1), λu+(1 - λ)v ∈ u ad . This stochastic system is a linear system, so it satisfies x λu+(1-λ)v =λx u +(1 - λ)x v . From this, we can obtain

[0028]

[0029] Therefore,[[]]

[0030]

[0031] Since λ ∈ (0, 1), when L(0, u)≥0, the above formula is greater than or equal to 0, satisfying the relationship L(x 0 , λu+(1 - λ)v)≤λL(x 0 , u)+(1 - λ)L(x 0 , v), so it is a convex function. When i = 1, 2, …, N, this conclusion still holds.

[0032] S3. Derivation of Riccati Equation

[0033] 1), Let the Lyapunov function \(V(t)=x^{\prime}(t)P(t)x(t)\), then by applying Itô's formula, we have:

[0034]

[0035] 2), To obtain the Riccati equation, we use the method of completing the square to deal with the weighted sum quadratic functional, and get:

[0036]

[0037] Therefore, the following Riccati equation is obtained:

[0038]

[0039] where \(P(t)\) is the unique positive definite solution of the above equation.

[0040] S4. Solution of the Pareto optimization strategy

[0041] 1), Considering the above Riccati equation, the weighted sum exponential cost functional can be expressed as:

[0042]

[0043] It can be seen that the optimal control rate is The weighted sum exponential cost functional affected by the optimal control rate \(u\) * can be expressed as

[0044]

[0045] where is the expectation with respect to given by the following formula.

[0046]

[0047] 2), Let We can obtain that \(G(t)\) satisfies

[0048]

[0049] Therefore, can be expressed as

[0050] In the above Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional, in step 3) of S1, the weighted sum method is used to assign weights to each objective function, including:

[0051] a. Judge the positive definiteness of the quadratic functional parameter matrix, or whether the minimum value of the quadratic functional under zero initial conditions is zero;

[0052] b. Give the weighted quadratic functional.

[0053] In the above-mentioned Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional, in step 2) of step S2, judging the convexity of the weighted sum quadratic functional includes:

[0054] a. Judge the convexity of the weighted sum quadratic functional.

[0055] In the above-mentioned Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional, in step S4, the operation process of the optimal value includes:

[0056] a. For the weighted sum exponential cost functional, apply Girsanov's theorem to eliminate the stochastic integral by taking the stochastic integral term in the Radon-Nikodym derivative as the stochastic integral;

[0057] b. For the optimal value of the weighted sum exponential cost functional, use G(0) to represent the exponential integral term therein.

[0058] In summary, the obtained Pareto optimal control method includes: Pareto optimal control strategy; the optimal value of the weighted sum exponential functional.

[0059] Compared with the prior art, the present Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional has the following advantages:

[0060] 1. The present invention applies Girsanov's theorem to eliminate the stochastic integral term in the exponential functional. The additional term appearing in the Riccati equation provides a suitable integral term for the exponential cost functional, making it form the Radon-Nikodym derivative. Through Girsanov's theorem, the Radon-Nikodym derivative eliminates the stochastic integral term generated in the exponential cost functional by the application of Itô's formula.

[0061] 2. The present invention generalizes the conclusion of a single-object single-agent system to a multi-object multi-agent system. Practical systems usually have the characteristics of multi-agents and multi-objectives, making the conclusion for a single-object system difficult to work. The application of the Pareto strategy solves the optimal control problem of the multi-object multi-agent system. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is the flowchart of the present Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional.

[0063] Figure 2It is the feedback control rate u 1 The change trend graph within time T.

[0064] Figure 3 It is the feedback control rate u 2 The change trend graph within time T.

[0065] Figure 4 It is the state vector x 1 The change trend graph within time T.

[0066] Figure 5 It is the state vector x 2 The change trend graph within time T.

[0067] Figure 6 Pareto front graph. Specific implementation manners

[0068] To better illustrate the present invention, the technical solution will be further described below in conjunction with specific embodiments and the accompanying drawings of the specification.

[0069] As Figure 1 shown, the operation principle of a Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional of the present invention is as follows:

[0070] In the case where both the quadratic functional and the admissible control set are convex, the quadratic functional is weighted and summed, and the multi-objective optimization problem is transformed into the problem of minimizing the weighted sum quadratic functional.

[0071] It is proved that the present invention relates to the convexity of the quadratic functional, so the weighted sum method can be used to solve the Pareto optimal solution.

[0072] The weighted sum quadratic functional is processed by the method of completing the square to obtain the Riccati equation.

[0073] The unknown matrix P in the Riccati equation is solved using MATLAB software to obtain the Pareto optimal strategy, which is substituted into the system to obtain the trend change graphs of parameters x(t) and u(t) respectively. By changing the weight parameter α, the Pareto front graph is obtained.

[0074] To more clearly illustrate the effectiveness of the above-mentioned Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional of the present invention, the above-mentioned control method of the present invention will be further described below in conjunction with specific examples.

[0075] As shown in the figure, a multi-agent multi-objective linear stochastic system with Brownian motion is selected as follows:

[0076] dx(t)=(A(t)x(t)+B1 (t)u 1 (t)+B 2 (t)u 2 (t))dt+D(t)dw(t)

[0077] where system initial state

[0078] The corresponding exponential cost functionals are respectively:

[0079]

[0080] and

[0081]

[0082] Let the weight α 1 = 0.2, then the quadratic functional after weighted processing is:

[0083]

[0084] where T = 1s.

[0085] For the above stochastic system and objective function, the solution steps of the Pareto optimal control method are as follows:

[0086] Input the parameters A, B 1 、B 2 、D, the initial state x 0 and the parameters involved in the objective function into the program. When the weight α 1 = 0.2, use the ode45 function in MATLAB to solve the Riccati equation, divide the time T into 1000 equal parts, and obtain the corresponding numerical values of P(t).

[0087] According to the optimal control rate obtain the feedback control rate and and their change curves within the corresponding time.

[0088] Substitute and into the stochastic system to obtain the closed-loop feedback system, and run to obtain the change curve of x.

[0089] Keep other steps unchanged, make the weight α change regularly from 0 to 1, and respectively obtain the corresponding numerical values of J 1 and J 2 to form the Pareto front graph.

[0090] 1. The present invention applies the Girsanov theorem to eliminate the stochastic integral term in the exponential functional. The additional term appearing in the Riccati equation provides a suitable integral term for the exponential cost functional, which forms the Radon-Nikodym derivative. Through the Girsanov theorem, the Radon-Nikodym derivative eliminates the stochastic integral term generated within the exponential cost functional by the application of the Ito formula.

[0091] 2. The present invention generalizes the conclusion of the single-object single-agent system to the multi-object multi-agent system. Actual systems usually have the characteristics of multiple agents and multiple objectives, making the conclusions for single-object systems difficult to work. The application of the Pareto strategy solves the optimal control problem of the multi-object multi-agent system.

[0092] The description and drawings are merely exemplary illustrations of the present application and are considered to cover any and all modifications, variations, combinations, and equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. A Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional, characterized in that: The following steps are involved: S1. Simplify and parameterize the linear random system with Brownian motion: 1) Select a linear random system with Brownian motion as follows: Where x0 is the determined initial state, (x(t), t∈[0,T]) is the system state vector, It's player j {j＝1,2，…，N} The control input vector, (w(t), t∈[0,T]) is defined in A standard Wiener process of suitable dimensions, where And T is a fixed value. Coefficient {A(t), B j (t), D(t)} is a known time-varying matrix of appropriate dimension. 2) The exponential cost functional of mutual coupling between players is: Assume that the quadratic functional in the above formula is Where Q i (t)>0, when i≠j, R ij (t)≥0,R ii (t)>0, etc. are all known time-varying matrices with appropriate dimensions, M≥0 is a known time-invariant matrix with appropriate dimensions, and E(·) represents the mathematical expectation. Let α i >0, i=1, ..., N satisfies so Exists. The weighted sum of the quadratic functional is obtained The optimal solution of the weighted and quadratic functional is the Pareto solution. 3) We set the joint control rate u(t) = col(u1(t), u2(t), ..., u N (t))∈R m and admissible control convex set U ad The simplified linear random system and weighted quadratic functional form is dx(t)=(A(t)x(t)+B(t)u(t))dt+D(t)dw(t) and Where B=[B1(t) B2(t) ··· B N (t)],R α =diag{R 1α , R 2α , …, R Nα }. Convexity verification of S2, weighted and quadratic functionals 1) The following conclusions are known to show that if U ad If and the quadratic objective functional are both convex functions, the Pareto efficient strategy can be obtained by solving the weighted sum optimal problem. Let α i >0, i=1, ..., N satisfies Assume that for i=1,…,N,U ad and the quadratic objective functional are both convex functions. If u * ∈U ad is Pareto efficient, then for all Pareto efficient u * There exists α such that The previous article has assumed that U ad It is convex. Considering that the objective function involved is an exponential cost functional and contains a quadratic functional, it is necessary to verify the convexity of the quadratic functional. 2) Prove that L is equal to i=1. i Convexity. Let x u is the state trajectory through the input variable u, and assume that u, v∈U ad , so we pass U ad Convexity inference for all λ∈(0,1), This random system is a linear system, so it satisfies x λu+(1-λ)v =λx u +(1-λ)x v . It follows that therefore, Since λ∈(0,1), when L(0,u)≥0, the above formula is greater than or equal to 0, and satisfies the relationship L(x0,λu+(1-λ)v)≤λL(x0,u)+(1-λ)L(x0,v), so it is a convex function. This conclusion still holds when i=1,2,...,N. S3. Derivation of the Riccati equation 1) Let the Lyapunov function V(t) = x′(t)P(t)x(t), then apply Ito's formula to get: 2) To obtain the Riccati equation, the weighted and quadratic functionals are processed using the matching method, and we get: Therefore, the following Riccati equation is obtained: Where P(t) is the only positive solution of the above equation. S4. Solution of Pareto optimization strategy 1) Considering the above Riccati equation, the weighted and exponential cost functional can be expressed as: It can be seen that the optimal control rate is The weighted and exponential cost functional affected by the optimal control rate u* can be expressed as in is given by the following formula about expectations. 2) Set We can get G(t) to satisfy therefore, It can be expressed as 2. A Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional as claimed in claim 1, characterized in that: In step S1 3), each objective function is assigned a weight using a weighted sum method, including: a. Determine the positive definiteness of the parameter matrix of the quadratic functional, or whether the minimum value of the quadratic functional under zero initial conditions is zero; b. Give the weighted quadratic functional.

3. A Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional as claimed in claim 1, characterized in that: In step S2 2), determining the convexity of the weighted sum quadratic functional includes: a. Determine the convexity of the weighted and quadratic functionals.

4. A Pareto optimal control method for a linear stochastic system with Brownian motion and an exponential cost functional as claimed in claim 1, characterized in that: In step S4, the optimal value calculation process includes: a. For weighted and exponential cost functionals, the random integral is eliminated by applying Girsanov’s theorem as the random integral term in the Radon-Nikodym derivative. b. For the optimal value of the weighted and exponential cost functional, G(0) is used to represent the exponential integral term.

Images