Method for solving an optimal control problem

Abstract

A method for determining, using a computing unit, an optimal control of a vehicle's powertrain for an optimal control problem, having a state function, initial state constraints, final state constraints, and a cost function, comprising determining, using the computing unit, a two-level optimal control problem, and solving said optimal control problem, said solution being determined by implementing a solution method using Pontryagin's maximum principle. Figure to be published: Figure 1

Description

Title of the invention: Method for solving an optimal control problem Technical field

[0001] The present invention relates to a method for solving an optimal control problem. It further relates to a computing unit configured to implement said method. State of the prior art

[0002] Methods are known for solving an optimal control problem such as that of minimizing the fuel consumption mFuei of a hybrid electric vehicle HEV, for Hybrid Electric Vehicle, over a given or predicted cycle of time T. A usual cycle used is the WLTC cycle (for Worldwide Harmonized Light Vehicles Test Cycle), lasting 1800 seconds, which is divided into four parts: a low speed zone for 589 seconds, a moderate speed zone for 433 seconds, a high speed zone for 455 seconds and finally a last one at "very" high speed which lasts 323 seconds and which is the most energy-consuming.

[0003] The control system is of dimension 2: the action levers, also called degrees of freedom, are the distribution of the torque between the thermal engine and the electric motor (defined by the selection of the thermal torque Tq and the control of the speed ratio (Gear selection).

[0004] The final State of Charge (SOC) of the battery must generally be fixed, as for standard homologation cycles requiring a SOC maintenance (or "iso-SOC") scenario.

[0005] The optimal control problem can be to determine: [oooô] T r min Tq (t), Gear(t)) dt

[0007] with: • the dynamics of the battery state of charge: SûdtW fh,SOC(t), Tq(i), GearOd • constraints on the evolution of the state of charge: SOCmh^SO <socmax • constraints on the commands, which can be seen as belonging to an admissible domain (which is not further developed here): ( Gear(t) j ^U(t) • constraints on the initial state and the final state of charge: SOC ( 0 ) = SOC^ SOC(T)= SOCT

[0008] The state of the art proposes several methods for solving optimal control problems, including in particular: • an equivalent consumption minimization strategy (ECMS), for the English Equivalent Consumption Minimization Strategy. It is an online method, but suboptimal. • direct methods: these methods require determining a discretization in time and solving an optimization problem in very large dimensions (these methods are based on the Karush-Kunh-Tucker (KKT) conditions) • dynamic programming methods: these methods require determining a discretization in time and exhaustively exploring all the possibilities of trajectory in state and using the Bellman principle (and exploring all the possibilities). • reinforcement learning methods: these methods require determining a discretization in time and control and using the Bellman principle (learning the Q function iteratively). • indirect methods: transform the optimal control problem (OCP) into a boundary value problem (BVP) using the Pontryagin Maximum Principle (PMP).

[006] Indirect methods are preferred because they provide a method that provides a much faster solution than dynamic programming methods, and they are much more accurate than other methods.

[0009] However, indirect methods implementing the Pontryagin maximum principle require the resolution of a low-dimensional optimization problem. This is generally determined by a "shooting method" based on iterative simulations of an internal model. But in the case of an application to cycles with a relatively long horizon, this method has some drawbacks: • possibly poor convergence or even divergence of the associated optimization problem, • too much sensitivity to initial values, • a potentially long calculation time, due to internal simulations over long horizons, • difficulties in taking into account state constraints (in the case of using HEVs: min-max limitations of the SOC)

[007] The present description aims to propose a method which at least overcomes one of the aforementioned disadvantages. Statement of the invention

[0010] To this end, according to a first aspect of the invention, a method is proposed for determining by a calculation unit an optimal control 11 of a vehicle motorization for an optimal control problem, having a state function x, initial state constraints xo, final state constraints xf, a cost function f°, what is written

[0011] (OCP) with : m in J y0 (t, x (t), u (t)) dt U 0 x(t) -f(t,x(t), u(t)) x(0) =¾ x(T) = Xf • x( t ) E the state, of dimension n, • u ( t ) E the control, of dimension m, * / ° ' ( ^ X ” X —> <y^la fonction de coût, • f : ( 'XX x the state dynamics function, • ; the initial state constraints, • xf: the final state constraints

[0013] According to the invention, the method comprises a determination by the calculation unit of a two-level optimal control problem, and a resolution of said problem of optimal control, said two-level optimal control problem being written:

[0014] (Macro) XM) Xo = Xq, XN+l = Xf (BOCP) V^X, XM) = minj / °(f, x(t),u(t)) dt ui, x(t) =f(t,x(t),u(t)) x(t / ) — X, x(tj+j) —

[0015] with: / o = O< ...< t,;< ... < tN+} — T

[0016] said solution being determined by implementing a resolution method using the simple shooting method of the Pontryagin maximum principle.

[0017] The single shot method is thus named in contrast to the multiple shot method.

[0018] Advantageously, prior to a resolution of said control problem optimal at two levels, the optimal passage costs V^Xj, from an initial state Xj to a final state XM are replaced by estimated passage costs C(X' X [) = V (Y' X j) predetermined before solving the problem of optimal control by the computing unit, the two-level optimal control problem (BOCP) becoming two independent problems, namely one problem optimization / Mq^To) ct + optimal order problems (Micro}

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] (Macro} (Microphone) Xi+1) Xo = x^ XN+l = Xf t 1 minx(t),u(t)) dt\ uh x(t) =f(t,x(t),u(t)) x(ji)=Xb x(tM)—Xi+l , According to one embodiment, the resolution steps / Macro} and (Micro} are made online. Preferably, the method further comprises a step of determining the estimated transition costs £* X j) = X- 0 predetermined from a set comprising initial states X^, final states Y(+1 and an optimal transition cost For example, each of the optimal transition costs Vz (Xp X / +î) from an initial state Xh to a final state Xj+l is determined by solving the optimal intermediate control problem OTP; ; Xi V^X^, XM) = minj / °(r, x(t),u(t)) dt x(t) = f(t,x(t), u(t)) x(tt)^Xb x(ti+i)-XM According to one possibility, the intermediate optimal control problem is solved by implementing a solution method using the Pontryagin maximum principle, preferably by a shooting method. According to a second aspect of the invention, a calculation unit is proposed for determining an optimal control of a vehicle engine, said calculation unit being configured to implement a method according to the first aspect of the invention, or one or more of its improvements. Description of figures

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] Other advantages and particularities of the invention will appear on reading the detailed description of implementations and embodiments which are in no way limiting, with regard to the appended drawings in which: • [Fig.l] represents, in a principle diagram of an exemplary embodiment of a method according to the invention. Description of embodiments The embodiments described below being in no way limiting, it will be possible in particular to consider variants of the invention comprising only a selection of the characteristics described, subsequently isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection comprises at least one characteristic, preferably functional without structural details, or with only a part of the structural details if this part only is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. In the figures, an element appearing in several figures retains the same reference. General description The invention addresses a method for solving an optimal control problem which can be written in its general form: T minJ / ° (t, x (t), u (t)) dt x(f ) — / ( x( fu(t)) x(0) =¾ x(T) —Xf with: • x ( t ) G the state, of dimension n, • u(t) G the control, of dimension m, • ' ( ^ x -''t" x —* X 'cost af°nc^on, • f : (X. xx 'a state dynamics function • possibly state constraints: x(f) G / , • possibly control constraints: u(t) GU (t) • xo ; the initial state constraints • xf: the final state constraints Transition to a two-level equivalent problem called (BOCP) An idea that is at the basis of the invention is to transform the optimal control problem (OCP) into an equivalent two-level problem, with one level, called (OCP) (Macro), presenting an optimal control problem, and another level, said (Micro), comprising a plurality of optimal control problems, from time intermediates on the interval [0, T] fixed beforehand.

[0034] This transformation can be written:

[0035] (Macro) Xi Xo = Xq. XN+l =

[0036] With / () = 0^ ... < ... < tN+i = T

[0037] The division into intermediate time can for example be carried out according to a predetermined period.

[0038] The transformation leads to iteratively solving N + 1 optimal control problems (Micro) over shorter intervals and then to introducing the resolutions of these shorter problems in the (Macro) part to solve the associated optimization problem.

[0039] Approximation of the problem (BOCP) and resolution

[0040] This resolution of the problem includes: • an offline stage, including: • a step of creating (El) a database comprising, for each of the time intervals A, = ti+ J, the 1; being fixed at prior, and for each of the initial states Xt and each of the final states Xi+h an optimal transition cost V^Xp XM ), then • for each of the time intervals A;- = [ / ,. a learning step (E2) of cost function C, from said optimal passage costs each of the cost functions C{ being a learning of the points V^Xj, XM). • an online step, including: • a resolution (E3) of the problem ÇMacro^, • a resolution (E4) of the problem (Micro) ■ Creating the database

[0041] To determine the optimal transition cost V) from an initial state X{ at time to a state final X(+1 at time ^+i, we solve the optimal control problem (Micro) to determine an optimal intermediate control trajectory, and we determine the intermediate passage cost associated with said optimal intermediate control trajectory.

[0042] For example, if the number of possible initial states at time is 11 and the number of final states at time ^+i is also 11, and the data creation strategy is an exhaustive exploration of the possible combinations, the database ultimately includes 121 optimal costs associated with a transition from an initial state to a final state.

[0043] There are techniques, known to those skilled in the art, for only having to explore the possible final states. There are other strategies where all possible combinations are not exhaustively calculated, even without defining a regular grid (other sampling methods) (example: Latin hypercube, random draws,

[0044] Since the optimal control problems (Micro) are independent, it is possible to solve them in parallel.

[0045] Determination of the cost functions G

[0046]

[20] The determination of the cost functions C, involves learning the dif different databases previously created.

[0047] Several methods can be implemented to determine the Q functions, and in particular: multilinear interpolation methods, which however require regularity of the database, approximation methods using polynomial functions, neural networks. Problem Solving (Macro}

[0048] The (Macro) problem of the (BOPC) is replaced by the resolution of the following (Macro) problem in which we seek the optimal intermediate states to minimize the sum of the estimated optimal costs:

[0049] . Xn+i-* /

[0050] Several methods can be implemented to find the intermediate states optimal for each time; • dichotomy method (only if the dimension is 1: N = 1), • gradient descent, • evolutionary algorithm to avoid encountering a local minimum, • brute force, • dynamic programming. Problem Solving (Micro)

[0051] The resolution of the problem (Micro) includes, for each of the intervals = [ tf , a determination of the optimal intermediate order solution of the following intermediate optimal control problem, using the Pontryagin maximum principle method of shots:

[0052] (Microphone) min J ^(t, x(t), u(t) ) dt U x(t) ~ f (t, x(t), ll(t)) x(ti+l)=XM Application

[0053] The present application proposes in particular an application to the resolution of an optimal control problem such as that of minimizing the fuel consumption mFuel of a hybrid electric vehicle HEV.

[0054] The optimal control problem can then be written:

[0055] y Gad'))dt

[0056] with • state dynamics: SÛC(t)=fsoc(t,SOC(t). TqICEÇ), GearM) • state constraints: 0.2 <soc(t) <0.8 • order constraints: ( ^ice^ Gear(t)}^U(t) • my initial and final constraints: SOC (0) = SOC^ SOC (T) = SOCT

[0057] We note that the problem is a very special case of the previous problem. It is sufficient to apply the method previously presented to solve it.

[0058] Of course, the invention is not limited to the examples which have just been described and numerous adjustments can be made to these examples without departing from the scope of the invention.

[0059] For example, it is envisaged to use the disclosed resolution method to determine optimal controls for the power distribution of a hydrogen vehicle.

[0060] In addition, the various features, forms, variations and embodiments of the invention may be combined with each other in various combinations. to the extent that they are not incompatible or mutually exclusive.

Claims

Claims

1. Method for determining by a calculation unit an optimal control 11 of a vehicle motorization for an optimal control problem (OCP), having a state function A, initial state constraints Ao, final state constraints xf, a cost function ce which is written: (OCP) T minJ / 0 (A x(r). u(t)) dt M o x(t) = f(t, x(r), u(t)) x(0) -xa x(T) =Xf with : • x( t) G the state, of dimension n, • u ( t ) G the control, of dimension m, • / ° ' ( 5^ X 5^” X the cost function, • f : ( X 5^” X 'Xm ) the state dynamics function, • Ao; the initial state constraints, • xf: the final state constraints involving a determination by the calculation unit of a problem of optimal control (BOCP) at two levels (Macro, Micro), and a resolution of said optimal control problem, said control problem optimal at two levels written:

2. (BOCP) with : (Macro) (Microphone) Xi+i) Xo= -¾ ^v+i xf h+ï Vi(Xb XM) = minf / °(r, x(t),u(t)) dt 1 x(t) — f (t, x(t), u(t) ) x(ti)=Xi, x(tM)=XM Zo — £0 ...< t^ ... < — T Vj(Xp X(+1) being the cost of moving from a state Xj to a state said solution being determined by implementing a resolution method using the Pontryagin maximum principle. Method according to the preceding claim, in which prior to a resolution of said two-level optimal control problem, the optimal transition costs V^Xp X+1 ), from an initial state Xt to a state final are replaced by estimated passage costs C (X- X- ]) = V (X- X ;) predetermined before solving the optimal control problem by the computing unit, the problem of two-level optimal control (BOCP) becoming two independent problems, namely an optimization problem (Macro I ct 1 optimal control problems (Micro} (E3, E4): (Macro'} (Microphone) X / +] ) . = ¼ XN+l = xf . min J ^(t, x( t), m( t) ) dt x(t) = f(t,x(t), li(t)) x(ti)=Xi, x(ti+i)=Xi+1

3.

4.

5.

6. Method according to one of the two preceding claims, in which the steps of solving the problems (Macro) and (Micro) are made online. Method according to one of the two preceding claims, when dependent on claim 2, further comprising a step of determining (E2) the estimated passage costs. C (X- X- j) = X- j) predetermined from a set comprising initial states X^ final states Xi+1 and an optimal transition cost V^X# Xi+1). Method according to the preceding claim, in which each of the optimal transition costs V^X,, Xï+1 ) from an initial state X^ to a final state Xr+1 is determined (El) by solving the intermediate optimal control problem: V^Xi, XM) = minJ / O, x(t),u(t)) dt x(t) =f(t,x(t), ll(t)) x ( h ) ~ x ( 1 ) — X;-+1 Method according to the preceding claim, in which the intermediate optimal control problem (Micro) is solved by implementation of a resolution method using the Pontryagin maximum principle.

7. Calculation unit for determining an optimal control u of a vehicle engine by implementing a method according to any one of the preceding claims.

Images